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Sampling methods, reconstruction methods, and devices for sampling 

and/or reconstructing signals. 

The present invention relates to sampling methods, related 
reconstruction methods, and devices for sampling and/or reconstructing 
5 signals. 

In many communication systems, the analog waveform is 
digitized as soon as possible, so that all further processing can be done in 
the discrete-time digital domain. 

In the prior art, such digitization requires sampling the 
10 continuous-time analog waveform at some regular interval called sampling 
period T. its dual, gv=2tc/T, is called the sampling frequency. Following the 
Shannon theorem, the signal to be sampled is band-limited to to m = wJ2 
with a band-limited filter The band-limitation Introduces a distortion of 
the signal, 

is !n other cases, a higher sampling frequency cos is used in order to 

satisfy the Shannon theorem without band-limiting the signal to be 
sampled. A high sampling frequency however requires fast expensive and 
power-consuming A/D converters, fast digital circuits and a waste of 
storage place for storing the digitized signal. 

20 An object of the invention is to find a sampling scheme for 

sampling at least some classes of non-band-limited signals, and for allowing 
an exact reconstruction of those signals. 

An other object of the invention is to find an improved method 
for samplmg at least some classes of band-limited signals with a sampling 
25 frequency lower than the frequency given by the Shannon theorem, and 
still allowing an exact reconstruction of those signals. 

An other object of the invention is to find an improved method 
for sampling signals which, even if they may be band-limited, can only be 
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seen through an imperfect measuring device having its own transfer 
characteristic. 

An other object of the invention is to find an improved method 
for sampling signals that uses uniformly spaced sampling times, and thus 
5 that is easier to Implement than methods using non-uniform sample 
intervals. 

According to the invention, those technical problems are solved 
with a new sampling method which allows sampling of a whole newly 
defined class of signals, including some non-band-limited signals, at a lower 
10 sampling rate than in the prior art. 

More specifically, those problems are solved with a new sampling 
method comprising the step of convoluting said first signal (x(t» with a 
sampling kernel (<j>(t)) and a regular sampling frequency (f), said sampling 
kernel and said sampling frequency being chosen such that the sampled 
15 signal (y s [n]) is a complete representation of said first signal (x(t)), allowing 
a perfect reconstruction of said first signal, wherein the sampling frequency 
(f) is lower than the Shannon frequency given by the Shannon theorem, 
but greater than or equal to the innovation rate of said first signal (x(t)). 

According to an other feature of the invention, the new method 
20 allows for a perfect reconstruction of the sampled signal. The reconstruc- 
tion method only needs knowledge about the class of the signal to 
reconstruct (for example a periodical stream of Dirac pulses, a piecewise 
polynomial signal, etc); it does not require any information on the signal 
itself, such as e.g. the times of the pulses or the position of the pieces in a 
25 piecewise polynomial signal. 

Applications of the Inventive sampling and reconstruction 
methods can be found in many technical fields, including signal processing, 
communications systems and biological systems. The sampling and 
reconstruction method can be used for sampling and reconstructing wide- 
30 band communication signals, such as CDMA signals, and ultra-wide band 
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communication signals, such as pulse-position modulated signal and time- 
modulated ultra-wide band signals, among others. 

The invention starts from a definition of a new class of signals 
which is broader than the usual class of band-limited signals. More 
5 specifically, the invention concerns signals with a finite rate of innovation 
p, i.e. signals having an (at least locally) finite number of degrees of 
freedom K per unit of time T. The sampling methods of the invention is 
based on the surprising finding that periodic or finite-length signals in this 
newly defined class of signals can be exactly specified by K samples, 

10 Various examples of sampling methods, adapted to different 

classes of signals and using various sampling kernels, are described in the 
following papers. In each case an exact reconstruction method which needs 
to solve only structured linear systems is described- The one skilled in the 
art will understand however that the invention is not limited to the specific 

15 examples given in this description, and that advantageous technical effects 
can be obtained by sampling other kinds of signals with a finite rate of 
innovation- 
Signals with a finite rate of innovation are very common. They 
include for example most signals produced by numerical circuits, D/A 

20 converters and processors, where the rate of innovation is limited by the 
clock rate of the electronic. They also include numerically produced audio 
and video signals. 

The invention will be better understood with the examples given 
in the following sections: 

25 The first section (slides) describes briefly with the illustration of 

block-diagrams the sampling and reconstruction methods used for various 
classes of signal with a finite rate of innovation. 



The second section (Sampling Signals with finite rate of 
innovation) gives a more forma! definition of the class of signals with a 
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finite rate of innovation or with a locally finite rate of innovation. It 
explains in more detail various sampling methods and reconstruction 
methods for signals with a finite rate of innovation. 

The third section (Sampling periodic signals with finite rate of 
5 innovation) describes sampling methods and reconstruction methods used 
for periodic signals with a finite rate of innovation. 

The fourth section (Sampling signals with finite rate and finite 
local rate of innovation) describes sampling methods and reconstruction 
methods used for finite length signais with a finite rate of innovation, and 
10 for signals with a finite local rate of innovation. 

The fifth section (Sampling of Plecewise Polynomial Signals) 
extends the methods to bidimensiona! signals, among others. 

The sixth and last section (Appendix 2.A) describes the 
annihilating filter methods which are used in various signal reconstruction 
is methods. 

The sampling and reconstruction method implies some 
calculations which can be done by a dedicated electronic circuit or, more 
conveniently, by a general-purpose processor or by a digital signal 
processor. The invention can be marketed as a product such as a dedicated 
20 circuit, a programmed circuit or a software product containing a program 
enabling a processing circuit to carry out the inventive method when said 
program is executed. 
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Claims 

1. Method for sampling a first signal (x(t» having a finite rate of 
innovation (pL 

said method comprising convoluting said first signal (x(t)) with 
5 a sampling kerne! (<j) (t)) and using a regular sampling frequency (f), 
said sampling kernel and said sampling frequency being 
chosen such that the sampled signal (y s [n]) is a complete representation of 
said first signal (x(t)} y allowing a perfect reconstruction of said first signal, 
characterized in that said sampling frequency (f) is lower than 
io the frequency given by the Shannon theorem, but greater than or equal to 
the innovation rate of said first signal (x(t)). 

2. Sampling method according to claim 1, wherein said first 
signal (x(tj) is not band-limited, 

3. Sampling method according to claim 1 or 2, wherein said first 
is signal (x(t» is a discrete-time signal. 

4. Sampling method according to claim 3, wherein said first 
signal (x(t» Is a periodic stream of weighted Dirac pulses, and wherein said 
sampling kernel (<f> (t)) is a periodic sine signal. 

5. Sampling method according to claim 4, further comprising a 
20 preliminary step of derivating said stream of pulses from an input signal. 

6. Sampling method according to claim 5, wherein said input 
signal is a periodical piecewise polynomial signal of degree R with K pieces 
per period, said preliminary step comprising differentiating R+1 times said 
input signal. 

25 7. Sampling method according to claim 2, wherein said first 

signal (x(t)) is a finite stream of weighted Dirac pulses, and wherein said 
sampling kernel ($ (t)) is an infinite sine signal. 
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8. Sampling method according to claim 2, wherein said first 
signal (x(t)) is a ffnite stream of weighted Dirac pulses, and wherein said 
sampling kernel (<j> (t)} is a Gaussian signal. 

9. Sampling method according to claim 1, wherein said first 
5 signal (x(t)) has a rate of innovation (pr) which is locally finite over a 

window of finite width (x). 

10. Sampling method according to claim 8, wherein said first 
signal is a bllevel signal. 

1 1. Sampling method according to claim 9, wherein said first 
io signal is a CDMA signal. 

12. Sampling method according to claim 1, wherein said first 
signal is a ultra-wide band signal such as a pulse-position modulated signal. 

1 3. Sampling method according to claim 8, wherein said first 
signal is derived from a piecewise polynomial signal. 

15 14. Sampling method according to one of the claims 9 to 1 3 t 

wherein said sampling kerne! (<|> (t» is a spline function. 

15. Sampling method according to one of the claims 9 to 14, 
wherein said sampling kernel (t)) is a box function. 

16. Sampling method according to one of the claims 9 to 14, 
20 wherein said sampling kernel (t)) is a hat function. 

17. Sampling method according to claim 1, wherein said first 
signal (x(t)) is a measure of a signal with a finite rate of innovation. 

18. Sampling method according to claim 1, wherein said first 
signal (x(t)) is a bidimensional signal 
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1 9. Sampling device comprising means for carrying out the 
method of one of the preceding claims. 

20. Computer program product directly loadable into the internal 
memory of a digital processing system and comprising software code 

5 portions for performing the methods of one of the claims 1 to 18 when said 
product is run by said digital processing system. 

21 . Method for faithfully reconstructing a first signal (x(t)) from a 
set of 2K samples (y s [n])), 

wherein the class of said signal to reconstruct (x(t)) is known, 
10 wherein the bandwidth I e> I of said faithful reconstructed 

signai x(t) is higher than co™ = n/T, T being the sampling interval, 

wherein the rate of innovation (p) of said faithful 
reconstructed signal is finite, 

characterized in that said method comprises the step of 
15 solving a structured linear system depending on said known class of signal. 

22. Method according to claim 21, wherein said first signal (x(t)) is 
a discrete-time signa!, said reconstruction method including following 
steps: 

finding 2K spectral values of said first signal (x(t)), 
20 using an annihilating filter method for finding said first signal 

(x(t)) from said spectral values. 

23. Method according to claim 21, wherein said first signal (x(t)) is 
a periodical piecewise polynomial signal, said reconstruction method 
including following steps: 

25 finding 2K spectral values of said first signa! (x(t)), 

using an annihilating filter method for finding a 
differentiated version (x R+1 (t)) of said first signal (x(t)) from said spectra! 
values, 

integrating said differentiated version to find said first signal. 
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24. Method according to claim 21, wherein said first signal (x(t)) is 
a finite stream of weighted Dirac pulses, said reconstruction method 
including following steps; 

finding the roots of an interpolating filter to find the position 
5 of said pulses, 

solving a linear system to find the weights of said pulses. 

25. Method according to claim 21, wherein said first signal (x(t}> is 
an infinite length signal in which the rate of innovation is locally finite, said 
reconstruction method comprising a plurality of successive steps of 

10 reconstruction of successive intervals of said first signal. 

26. Method according to one of the claims 21 to 25, wherein said 
first signal (x(t)) is a tridimensional signal. 

27. Method according to one of the claims 21 to 26, wherein said 
first signal is a measure of a signal with a finite rate of innovation, said 

15 method being used for reconstructing the original signal before measure. 

28- Method according to one of the claims 21 to 27, wherein said 
first signal is a bilevel signal. 

29. Method according to one of the claims 21 to 28, wherein said 
first signal is a CDMA signal. 

20 30. Circuit for reconstructing a sampled signal by carrying out the 

method of one of the claims 21 to 29. 

31. Computer program product directly loadable into the internal 
memory of a digital processing system and comprising software code 
portions for performing the methods of one of the claims 21 to 29 when 
25 said product is run by said digital processing system. 
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Sampling signals with finite rate of innovation 

Martin Vetterli 1 * 2 Pina Marailiano 1 Thierry Bhi 4 ' 

Abstract 

Consider tha problem of sampling signals which are not ba^dlimifced, but still nave 
a finite number of degrees of freedom per unit of time, such as* for example, piecewise 
polynomials Call the number of degree of freedom per unit of time the rate of innovation. 
We ti^onstatte that by uglng an ad&tjuate sampling kernel and a sampling rate greater or 
equal to the" rate of innovation, om cart. uniquely reconstruct such signals. 
We thus proves theorems for dosses of signals and sampling kernels thafc generalise the dasaic 
"bandlisnitsd and sine kernel* 1 case- In particular, we show sampling theorems for periodic 
as well as finite length piecewise polynomials, using a bandlimited derivative kernel, as well 
. as a Gaussian kernel. Bbr infinite length piecewise polyhamials with a finite iocal rate of 
Innovation, show exact local reconstruction using sampling with spline kernels. 
Ail the results presented lead to computational procedures that are readily implemensahte, 
which is shown through experimental results. Applications of these new sampling results . * 
■ can be found La signal processing, communications systems and biological systems. 

Index Terms 

Sampling, generalized sampling, poisaon processes, piecewise polynomials, non-bandlimited 
signals, analog-to-digital conversion, annihilating filters. 
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SAILING SIGNALS WITH FINITE RATE QF* INNOVATION 



L INTRODUCTION 

Most continuous-tinfe phenomena^ can only be seen through sampling tha continuous- 
time waveform, and typically, the sampling is uniform. Very often, instead of the waveform 
itself, otic has only access to a smoothed or filtered version of it. This raay be due to the 
physical set up of the measurement, or may be by design, 

Calling %(t) the original waveform, its filtered version #(£)*<£(t), where <p{i) = is 
the convolution kernel Then, uniform sampling with a sampling internal T leads to samples 
x[n] given by 

OCf 

x[n]**<<p(t-nT),at(t)>= j ip(t - nT) x{t) dt. (1) 

— OO 

This setup is shown in Fig. 1. 



x{t) 





Vs\n} 



Fig. 1. Sampling set up: x{t) is the contmuous-time signal; j>(±) = is the smoothing kernel; y(t) is , 

the filtered signal; T is the sampling interval; y* [n] = a?[»], n e & are the sample values. 

When no smoothing kernel is used, we simply have x[n] = x(nT) f which is equivalent to 
(1) with tpi*) ~ *(*)■ This simple . model for having access to the continuous-time world is 
typical for acquisition devices in many areas of science and technology, including scientific 
measurements, medical and biological signal processing and analog-to-digital converters. 
The key question is of course if the samples x[n] axe a faithful representation of the original 
signal x{t). If so, how can we reconstruct %{t) froms[rc], and if not, what approximation x {t) 
do we get based on the samples x[n]? This question is at heart of signal processing, and the 
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SAMPLING SIGNALS WITH FINITE EATE OF* IHNOVA.ttON 4 

dominant result is the well-known sampling theorem of Whittaker, Kotelnikov and Shannon 
Which states that if is baaidtimited, or === 0, |a>i > w m » then samples #[nj = #(rtjf*) 
with J 1 < 7r/o>rrt are sufficient to reconstruct %{t) [4|* [6]» [9]. The reconstruction formula is 
given by 

*(ft) * ]T ff(n:T} sinc(t/T - n), . (2) 

with 

amc{t) — (3) 

If isf not bandliitntedj convolution with <p(t) — einc(t/T) (an ideal lowpass filter with 
support [— 7t/T>ir/T]) allows to apply sampling and reconstruction of x{t) % the lowpass ap- 
proximation of x(t), or the restriction of X (w) to the interval [— ?r/T, fr/Tj. . 

A possible interpretation of the interpolation formula (2) is the following. Any rfcat ban- 
dlimited signal can be seen as having 1/T degrees of freedom per unit of time, which is the 
number of samples per unit of time that specify it. In the present paper, th}s number of 
degrees of freedom per unit of time is called the rate ot innovation of a signal 1 , and 
is denoted by p. In the bandlimited case above, the rate of innovation is p = 1/jF = w m /7i\ 
In the sequel, we are interested in signals that have a finite rate of innovation, either on 
intervals, or on average. Take a Poisson process, which generates Diracs with independent 
and identicaily distributed (LLd.) interaxrival times, the distribution being exponential with 
probability density function /ie w|ltt . The expected interarrival time is given by Thus, 
the rate of innovation is fi 7 since on average, fi real numbers per unit of time fully describe 
the process. 

Given a signal with finite rate of innovation, it seems attractive to sample it with a rate of 

1 This is different from the rate used in rate- distortion theory [3j. Here, rate corresponds to a degree of freedom 
that are specified by reel numbers. In rate-distortion, the rate corresponds to bits. 

March 23, 2001—3 : 38 pm DRAFT 



SAMPLING SIGNALS WITH BUtfTB RATE OF INNOVATION 5 

p samples per unit of time* We know it will work with bandlimited signals, but will it work 
with a larger class of signals? Thusj the natural questions to pursue are the following: 

1 . What classes of signals of finite rate of innovation can be sampled uniquely, especially 
using uniform sampling ? 

2. What kernels y>(t) allow for such sampling schemes? 

3. What algorithms allow the reconstruction of the signal based on the Samples? 

In fclie present paper, we concentrate on stream of Diracs and on piecewise polynomials, 
which are classes for which we are able to derive sampling theorems under certain condi- 
tions. The kernels involved are related to the sine (the bandlimited derivative) i the Gaussian, 
and the spline kernels. The algorithms, while more complex than the standard sine sampling 
of bandlimited signals, are still reasonable (structured linear systems) but also often involve 
root finding. . 

The outline of the paper is as follows. Section II formally defines signals with finite rate 
of innovation. Section III and Section IV consider periodic signals in discrete and contin- 
uous time respectively, and derive sampling theorems for streams of Diracs and piecewise 
polynomials. Both of these type of signals have a finite number of degrees of freedom, and 
a sampling rate that is sufficiently high to capture these degrees of freedom, together with 
appropriate sampling kernels, allows perfect reconstruction. Section V addresses the sam- 
pling of finite length signals having a finite number of degrees of freedom, using infinitely 
supported kernels like the sine kernel and the Gaussian kerneL Again, if the critical number 
of samples is taken, we can derive a sampling theorem. Section VI concentrates on local 
reconstruction schemes. Given that the local rate of innovation is bounded, local reconstruc- 
tion is possible, using for example spline kernels. Finally, Section VII derives applications of 
the abovfc results, in particular to piecewise bandlimited signals, and to filtered streams of 

March 23, 2001—3 :38pm ' DRAFT 



SAMPLING SIGNALS WITO WNttB HATE OF INNOVATION 6 

In the Appendix, introduce the "aimihitating filter method borrowed from spectral anal- 
ysis* This method will be referred to in all of the proofe in Section III, Section IV as well as 
one in Section V-B. 

II. Signals Witrt finite rate of innovation 

In the introduction* we have informally dkcuased the intuitive notion of signals with 
finite rate of iimov&tiotL More formally, consider functions or signals having a parametric 
representation. 2 Then: 

Definition 1: The t&te of innovation p is the avef&g£ number of degrees of freedom per 
unit of time j or ? with C^to, *i) giving the number of degrees of freedom of x[t). over the 
interval [to,ti] s 

P -toicc-I.t). (4) 

Definition 2: A signal with finite rate of innovation is such that p < oo. 
If v/e consider finite length or periodic signals of length r, then the number of degrees of 
freedom is finite, and the rate of innovation is 1/r C 9 [0,r]. 

Bandliinited signals with frequency support [— ir/T, tt/T] have a rate of innovation p = 
1/T since they are uniquely specified by samples taken every T seconds. Bandpass signals* 
having support {—ujq — A, — w 0 ] U l^u^ .+ A) have a rate of innovation p = A/r, since 
appropriate demodulation allows sampling with a period T == 2tt/2A. 
If we consider discrete-time sequences, then general sequences have a (normalized) rate of 
innovation of 1 (one degree of freedom per sample) , If the underlying sampling rate is taken 
into account, then we have again a rate of innovation p = 1/T. 

A stream of Diracs in discrete-time with K locations over an interval of size N has a rate 

* In the sequel, we consider real signals unless specified otherwise. 
March 23, 2001— 3; 38 pax DRAFT 



SAMPLING SIGMALS WITH FINITE ftATS OF tNNGVATlON 7 

of innovation of the order K/N * but these are degrees of freedom over the integers. A 
piecewise polynomial in discrete-time has thus a combination of integer and real degrees of 
freedom. 

One can. also define a local rate of innovation with respect to a moving window of sise r. 
Definition 3; Given a window of size r # the local rate of innovation at time t is 

fr(*)*"7<4(*-*-/ai*+t-/2). 

In this case, one is often interested in the maximal local rate, or p^{r) 

As r -> co, ^(r) tends to p. To illustrate the differences between p and p mi consider again 
the Poisson process with expected Lnterarriv&l time t/p. The rate of innovation p is given 
by fM. However, for any finite r, there is no bound on PntCO, even though its expected value 
is /i. • 

While one can define many parametric signals which have a finite rate of innovation, in the 
sequel we will concentrate on streams of Diracs and piecewise polynomials which axe classes 
for which we are able to give sampling theorems and reconstruction formulae. 
Combinations of bandlimited signals and piecewise polynomials are also of interest, as are 
filtered versions of stream of Diracs. 

Ill, DISCRETES TIME PERIODIC SIGNALS WITH FINITE RATE OF INNOVATION 

The discrete-time periodic signals we consider are streams of weighted Diracs and piece- 
wise polynomials. Through appropriate differentiation, piecewise polynomials can be reduced 
to streams of Diracs, so we begin with these. 

3 Actually, slightly less because there can only he one Dkac at any one location. 
March 23, 2001-^3 r 38 pm DRAFT 
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A. Stream of Dimes 

Consider a disdtetfc-time periodic signal, with one period given by 

x = MOUM- A*f-i]f (5) 

and containing K weighted Dimes at locations {n^ni,... ,njr_i}, n k e [Oj 2V" - 1] and 
K<[N/2\ t . 

/c-t 

X N = $2 c **[ n ~ n *K (6) 

where 5(n] is the JCronecker delta and equal to 1 if n — 0 and 0 if n £ 0. 
Denote by X « (X[0],X[1],. . . , JT[jV - 1]) T the disctete^time Fourier series (DTFSJ coeffi- 
cients of x where 

X[m] - J2 C * W *™> m = 0,-..JV-"l (7) 

Consider filtering the signal with a lowpass filter (p[n\ — <p[— rc] with, bandwidth [— if, JST] 
then the sample values y^f] are simply a subsampled version {by M) of the filtered signal 
y[n] = * <£jn]. The DTFS coefficients oiy[n] are given by 



if m 
0 else 



and those of the subsampled signal = y[lfyf[ are given by the usual subsampling formula 

y,n^~^2Y[{m + lN)/M). (9) 

With appropriate re-indexing it follows that 

y 3 [m] = -Lx[m\ t m£[-K,K\. (10) 
M. 

March 23, 2O0l-3:3ftpm DRAFT 



SAMPLING SIGNALS WITH FINITE RATE OF INNOVATION g 

Figure 2 illustrates that we can recover IK spectral values X[tn] of the original signal from 
the subsampled spectra of the lowpasS approximation Y„[m) as long as there is no overlapping 
in the spectra of the lowpass approximation Y[tn] aid this occurs only if N/M > IK. 




(a) 









\ 


. 




j 








(b) 




v> 






\ 










« • 1 



(c) 

Pig, 2, (a)DTFS of stream of Diracs, X\k\,k 6 [0, JV]; (b) DTFS of lowpaae approximation r[fe] - X[fe], fc € 
[-#\#},D otherwise; (c) DTFS of lowpass approximation subsampled by M = 3, K B (fe} = l/MX[fe] 3 fe e 

. This leads us to 

Proposition 1: Consider a diBcrete-tiine periodic signal zfcjof period iV containing K 
weighted Diracs. Let M be an integer divisor of N satisfying N/M > 2K + I. Consider 
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SAMPLING SIGNALS WITH FINITE HATS OF INNOVATION 10 

K 

the discrete-time periodica sine sampling kernel tp[n] = i £ W£ mr \ that is, the Inverse 
DTPS of the Rect^rcK]. Then the iV/Af € N samples defined by 

VM - < x[n}i <p[n - IM] > circ) t = 0, . . . . JV/M - 1 (n) 

are a sufficient representation of the signal. 

Proof: We start by showing that the DTFS coefficients X[m], m e [- jfif, #] are sufficient 
to determine the stream of K weighted Diracs . Then we show th&t the Pf/M sample y*[t] 
axe a sufficient representation of X[mj 4 m € [— K s f£\. 

1. Since X[rn] is a linear combination of K complex exponentials, v#\ with u k = 
the locations .re* of the Dixacs catx be found using the annihilating filter method 
. described in Appendix A. It suffices to determine the annihilating filter: H{z) whose 
coefficients are (1, #[1], . • . , H[K]) or„ , ' 

J3f (a) = I 4- H[l] + H[2] z~ 2 + - . • + H[K] sT K . (12) , 

which factors as 

K-l 

H{z)=T[{l-z- l W%<) (13) 

and satisfies 

k " 

Y^H[k]X[™>-k}=Q> m = 0,... >JV-1 ■ (14) 

Since Zf[G] = 1, equations (14) will be sufficient to determine the K unknown 
filter coefficients H[k], k = 1, . . . , K . Let m — 1, . . . , K then the system in (14) is 
equivalent to 

K 

J2B[k]X[m-k}^-X[m] f m = !,..• , K; ' (15) 
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1 M »^ 



SAMPLING SIGNALS WITH FINrTSS SATE 01? INNOVATION 



11 



X[0] X[-l] X{-2] 
X[l] X[0] X[-l] 
X[2] X[l] X[Q] 



For example take N = 8, IC = 3 Arid let m =a 1, 2,3 then in matrix/vector form the 
system is 

£T(2] = - X[2) j . (16) 

\ m ) \ m j 

Given that these are K ssnueoids the matrix in (16) is full rank (== K) and thus there 
is a unique solution H{1], .... H[K\. The set of locations {««,, n u . . . , nj<-_ t } are given 
by the the zeros of H(z). 

The weights of the Diracs are obtained by solving K equations in (7), let. m — 
0, . . . , K — 1, this leads to the following Vandermcnde system 



1 

'7 



/ co \ ( X[Q] \ 



and has a unique solution since the ti* ^ n f) V& ^ Z, 
Therefore, given 2iT contiguous DTFS coefficients 



X[l] 



(17) 



we have found a unique set of locations {tc*}]^ 1 and a unique set of weights {cjb}^ 1 * 
2. We need to show that 2K spectraj values X[m],m E I^if, AT] can be obtained from 
the N/M sample values y s [l] defined in (11). 

We substitute the discrete-time pmodised sine kernel in the expression of the sample 
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values and we obtain the following: 



Vs[i\ - <x[n],tp[n-lM] > erfrc I=t 0, . . . % N/M - 1 (ig) 



U-X K 



= il>N 2 w^-" 0 (20) 

- E wy**^ 1 * (2D 

= y E E ■M W U nm (22) 



1 * 



= Jf H X[-m]W^j M . (23) 

If we calculate the DTFS coefficients of the sample values y s [l] we obtain an expression 
in terms of the DTPS of the signal, 

YmW - £ tf.WWjf/i£,. fc«0 i ... f N/AT-l (24) 
- 7j E E ^I-«]WS^WJ?/m (25) 

= 4e ^i-h e <r } (26) 



■i 



JV/Af ifjfe.+ m = 0 
0 • otherwise 



= j^Xlk}, k = 0,... ,wn{K,N/M-l} (27) 
M 

^X[k] = MTsW, fe = (28) 

by Hypothesis, N/M > 2K + 1 > K. Since we are dealing with reel signals the DTPS 
is Hennitian s that is, X[-k] = X*[k], k = 0, . . . >K> so we have the 2K Hh 1 spectral 
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values X[k) f k e [-K,fc] obtained from the NJhf DTPS coefficients of the sample 
values y B [l\. Therefore we have a sufficient number of spectral values which uniquely 
define the stream 'of weighted Diracs. 

■ 

Figure 3 illustrates in time and frequency domain the sampling of a discrete-time periodic 
stream of Dimes with period N ^ 256 and K ~ 15 weighted Diracs. The signal is perfectly 
reconstructed within machine precision, MSB = lCT 11 . 

Note that in the proof of Proposition 1 the locations of the Diracs ate determined by finding 
the roots of the annihilating filter B{z). If the locations are bunched up or there ate a large 
number of Diracs then finding the roots of the polynomial is numerically unstable. An 
alternative method that is commonly used in error correction coding involves extrapolating 
the N—K spectral values of the signal using K first spectral X[k), k = 1*, „ * , K components 
and the error locating polynomial which in our case corresponds to the annihilating filter 

K 

X[k\ = -^H[l[X[k-il, k = K + l,„. } N^K. (29) 

Consider a signal of length N = 64 where there are K — 16 Diracs in an interval of size 2K t 
see Figure 4. Figure 5 compares the relative reconstruction eTroT between the root finding, 
method and the spectral extrapolation method for different values of K . 

B. Piecewise polynomials of degree R 

The previous result on the stream of Diracs is extended to piecewise polynomials. Con- 
sider a discrete-time periodic piecewise polynomial defined by 4 

4 n + = n 3 if n > 0, and 0 else. . 
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of period N with K pieces each with maximuai degree R* Suppose a discrete-time difference 
operator d[n] « S[n] — 6[ft — 1] is applied R + 1 times to the piecewise polynomial signal. 
The differentiated signal ^ +1 [n] in frequency domain is 

Xt*+V[m] (D[m])^ } m - 0, . . . , N - 1 (31) 

where £>[m] = 1 — is the DTFS o£ the discrete-time difference operator. This results in 
putting to zero ad! the polynomial pieces. Assume there" &re discontinuities between pieces 
(but no Diracs)/ then tC transitions can lead to afc most K{R^- 1) weighted Dirass and 
thus the rate of innovation is p = 2K{R + l)/iV. From Proposition 1 we can uniquely 
recover the K{R 4- 1) Dfracs frotn 2K(R + 1) DTFS coefficients of the differentiated signal 
Xt R +V[k]. The piecewise polynomial signal is reconstructed by applying the Inverse discrete- 
time difference operator R '+ 1 times on the stream of weighted Diracs. The discrete-time 
difference operator d[n] is a singular operator (since D[Q] = 0) and so we define the inverse 
discrete-time difference operator as D^rn] = 0 for m ~ 0 and D^fm] = (1 ~ W^)" 1 for 
ro^l,. Hence instead of using the sine sampling kernel <p[n] we will use -the 
derivative sine sampling kernel defined by tj)[n] ~ ( 4*d*-' *£ *cp)[n] which has at least 
R+l zeros at the origin z = 1. Then the DTFS of ^[n] is . 

^{ m ] = {l-W^)* +1 *[m], m-0,..< (32) 

where $[m] is the JHcrt[-jc(ji+.i) 1 ir(B+j)i function. This brings ns to the following theorem. 

Theorem 1: Consider a discrete-time periodic piecewise polynomial signal of period N 
with K pieces of degree R and with zero mean. 5 Let M be an integer and a divisor of N 
such that N/M > (2#(i? + 1) + 1). Take a sampling kernel tfn] with DTFS coefficients 
defined in (32). Then we can recover the signal from the N/M G N samples 

y.ffl - <*[nl f ^i.-U*]> I^0,...,JV/M~1. .(33) 

s We consider zero meat signals since £> o D~ l is a projector on tke space of. signals having zero mean. 
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Proof: Start we show that 2if{£ + 1) DTFS coefficients of the signal, X[m],tn € 
[-if 4- 4 1)] are sufficient to determine the piecewise polynomial signal, x[n]. 

Then we show that the JV/Af samples ^[1] are sufficient to determine the {2K{R + 1) + i) 
values jffmj. 

1. If we haw the DTFS coefficients #[m],m € + 1),K{R+ 1)] then from (31) 
we have the DTFS coefficients of the {R + l)th discret^time differentiated -signal, 
^(Ji4-i)| m ] t p tom proposition 1 these are sufficient to reconstruct the Stream of K[R+ 
I) Dira^SL Thus, the signal is recovered by applying times the inverse disctete- 
time difference operator, dr l [n] t on th£ stream of Dirace j that is, 

x[n] = ♦ <r* ♦ - v ♦ d-^ gtW))^], 

2. Similar to the second part in the proof of Proposition 1 we expand the inner product 
between the piecewifce polynomial signal and the differentiated ainc sampling kernel: 



VM = <x[n} > ip[n-tM]> 7 i » 0, . . . 7 N/M - 1 (34) 
■2Z*f n M n -- | * f l (35) 



- ^E^n E a - v^ m(n - ,ltf5 . (36) 

- N-l rc(n+i) 

= ]yX>M 2 (l-^) H+i ^ mn W^^ (37) 

n=0 jn=-K{R+l) 

. JE"(B4-I) N-I 

= ^ E (l-^) s+1 X[-m]^. (39) 
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Taking the DTFS of the sample values we obtain 



2 fcMwft*. *-o iv/Af-i 

t Ar//tf-i /c(J?.+i) 

1=0 m=-#(*Hl) 

£ E. d-^) R+i ^-m] x: wssr* 



7V/A<f if fc + m = 0 
0 otherwise 



16 



(40) 
(41) 

(42) 



| 0 fo 



<r *« 1,.,. ,J5f(fl.+ l) 
for k 0 



(43) 
(44) 



Since JV/Af > (2/C(i2 + 1) + 1) we have a sufficient representation for the spectral 
values of the signal. This completes the proof. 



Figure 6 illustrates the reconstruction of a discrete-time periodic piecewise linear signal 
of period N = 1024 with K - 6 pieces. 

IV, Continuous-time periodic signals with finite rate of innovation 

We derive now the equivalent results but for continuous-time periodic signals, again 
building up from & stream of Diracs to piecewise polynomials. We will put in evidence the 
common points. 



March. 23, 2001 — 3 : 38 pro. 



DRAFT 



SAMPLING SIGNALS Willi FINITE RATE OF INNOVATION 17 

A. Stream of Dimes 

Cotiaidet acotttijiuotig-tinie periodic Signal or period r containing K weighted Diracs 
at locations {ik} 1 ^ With i k € [Q, r), or 

I22Z CA ^-c*fe+^)) (45) 

since t ti + K ^U+r and == for all ft g N- 

The con£inxious-titne Fourier series (CTFS) coefficients of *(f ) are defined by 

r 

,Y[ m ] = ±Jx{t)e- mifn ' r dt, ruGZ 
o 

= ^E c ^^ Wr " (46) 



If the signal is convolved with a sine filter of bandwidth [-K, K] then we have a lowpass 
approximation y{t) given by 

K " 

y[t) = Xltn]e <w ^ (47) 

Suppose the lowpass approximation y(t) is sampled at multiples of T 3 we obtain r/T € N 
samples defined by 



y*[i] = y(iT) - E x N e " r/T > i-o,...,r/r^i. (48) 

Similar to the discrete-time case as long as the number of samples is larger than the number 
of values in the spectral support of the lowpass signal, that is, ^ > 2K + 1, (48) can be used 
to recover 2K + 1 values of X[m], Thus we can state: 

Proposition 2: Consider a continuous-time periodic stream of K weighted Diracs with 
period r and a continuous-time periodic sine sampling kernel <p(t) with bandwidth [— iff, if]* 

March 23, 2001—3 : 38pm DRAFT 



SAMPLING SIGNALS WITH PWITB liATEi OF INNOVATION ig 

Taking a sampling period T such that r/T £ N and r/T > 2tf + 1; Then the samples 
defined by 

y s \(\ - <x{t) i <p(t~lT)> j 2^0,..- .r/F-l (49} 
are a sufficient representation of #(f ). 

Proo/r Similar to the discrete-time case first we show that 2jFC + 1 CTFS coefficients 
X[m] are sufficient to find the locations and the weights of the Diracs. From (46) we have 
that the OTPS coefficients X[m\ are linear combinations of complex exponentials. Tints to 
Bnd the locations t k we need to find the annihilating filter H «. ff[2], . . . >H[K]) ' 

such that 

H* C X = 0. (50) 
This is the same Toepiitz system as in (14) considered in Sec. Ill- A and therefore a solution 

K 

exists. Factoring the z— transform of H 7 or H{z) = £ H[k] z~ k , into 

H(z) - - ( 51) 

we then find the K locations {to, ti,--. . , from the zeros of H[z)^ that is, from 

u k - e ^^ T . . ' (52) 

Given the locations {fo}^ 1 and j<f values I[m],m = 0,.,. ,-ftf — 1, we iind the weights 
{ c A:}fci l of the Diracs by solving the Vanderrnonde system in (46). Since the locations t h are 
distinct, f & ^ 4 fj ^ £, the Vanderrnonde system admits a solution- 

The second part of the proof consists in showing that the r/T samples y#[l] are sufficient 
to determine the CTPS coefficients JC[m],m € {— if, if]. We substitute the continuous- time 
periodic sine function #(t) with bandwidth [— K, K) defined by 

K 

wit) - < 53 ) 
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in (49) and we obtain 

y$) = < x{t)^{i-lT) >, i«0 } ... t r/T-l (54) 



(55) 
(56) 



0 



/C 

m T 12 X[^m\e^ M f\ (57) 
"Note that is periodic with period r/T, thus the DTPS coefficients are.K^fe] = TX[k], k = 
.0, . . . 7 r/T — 1, Since r/T > 2iT-h 1, tye Lave a sufficient number of samples that determine 
the CTFS X[m],m el-K, l<]. m 

B. Piecewise polynomials of degree R 

Here we consider contiguous-time periodic piecewise polynomial signal of period r a con- 
taining K pieces of maximum degree R and R — 1 continuous derivatives, C R ~ l , 

* W = pi (* " f * e [°> T ± " <5« 

We differentiate the signal i2-f 1 times and we obtain a continuous-time stream of K weighted 
Diracs, x^ l \t). The OTPS of the derivative operator is defined by D[m) = i2nm,rn E Z 
and therefore the CTFS coefficients of the differentiated signal ar? R+l3 (t) are equal to 

X^[m] - (£2™)*" X[m] 9 ™e Z. (59) 

From Proposition 2 we can recover the continuous-time periodic stream of K Diracs from 
the CTFS coefficients, X^ R ^[m),m e [~K X K]. Therefore we can sample the signal with, 
the differentiated sine sampling kernel wliose CTFS coefficients are defined by 

®[m] — flEwm)** 1 m £ Z (60) 
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SAMPLING SIGNALS WITH FINITE RATE OP INNOVATION 20 

where *[m] ~ Beefy-.*,]?) is the OTPS of the continuous-tittie^petiodked sine sampling kernel 
Theorem 2: Consider a continuous-time periodic piecewise polynomial signal %{t) with 
period r* containing K pieces of maxirnurn degree R, belonging to C*"* 1 and havitxg zero 
mean, Consider a sampling kernel with its CTFS coefficients defined in (90). Let 
r/T e N and r/T > 2K* + !*■ Then s{t) can be uniquely recov&<*d from the r/T samples 

V M -< *(*), tf(t -«*)>, I « 0, . • ; f r/T - 1. (61) 
Pttto/; Similar to the proof of Theorem. 1, we first show that OTPS coefficients X[m] , m e 
[— K % K] are sufficient to determine the piecewise polynomial signal, x(i). Then we show that 
the r/T samples p$] are sufficient to determine the values X[m], m g [— K, K\, 

L If we have the CTFS coefficients X[m]< m e [— if > K ] ' then from (59) we have the 
CTFS coefficients of the (K+l)th differentiated signal, X^ 1} [rn]. Prom Proposition 2 
these are sufficient to reconstruct the stream of K Diracs. Thus $ the signal is recovered 
"by integrating R + 1 times th t e stream of Dixacs, that is, 



x(i) - J J - - - J x^ l) (t) dtdt--dt 



or in frequency domain from (59) 



X[rn] = {D~ l [m]) R+1 X&+V{rn], m € Z/{0} * (62) 
= («2arm)-<^ 1) X^ 1 >(m] t m € £/{Q} (63) 

with D^ro] 0 for m = 0 and thus 

2: Similar to the second part in the proof of Proposition 2 we expand the inner product 
between the piecewise polynomial signal and the diiferentiated sine sampling kernel 
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ic 

defined by 0(f) ^ £ ^itm)*^ 2 ***^. That is, the sample values are 

«bW =* < #(*)^(* - i5T) > 4 i = 0, . . . , r/3 1 - 1 (64) 



(66) 



~ 2 (»2trm) R+l e -«^"-/- /" e «*»**/* ^ (66) 



- r S ^hw](i2^m) a+1 e-^/^- (67) 
Since is periodic with period t/I\ the DTFS coefficients of # S |T| are given by 

= X y 9 [l}e" i2tktf{rm (68) 
= r]£ £ *[-7n] ^m)^ 1 er****ti*m 

(69) 

K r/T-1 

■ • s »„ " 

f r/T if fc + m~G 
[ 0 otherwise 

« ^-ftrm)«x[ t ] (71) 

Therefore the CTFS coefficients of the signal axe obtained by the DTFS coefficients 
of the samples values Y 5 [mj, m = 0, , . . , t/T *— 1 and axe defined by 



x , , = , - - *H/V (-a«ii)» l ) for m = 1, . . : , r/T - 1 ' 
J ^ % 0 for m - 0 



Since t/T > 2K + 1 the sample values are a sufficient representation of the spectral 

values of the signal. This completes the proof. 
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■ 

Note that removing the restriction z(t) € C*" 1 leads to the same result as in Theorem^ 1, 

V. FINITE LENGTH SIGNALS with finite hates op innovation 

A. finite length signal with finite rate of innovation /> clearly has a finite number of degrees 
of freedom. The question of interest is: Given a sampling kernel with infinite support^ is 
there a finite set of sample* that nniquely specifies the signal? In the following sections 
we will sample signals with finite number of weighted Diracs with infinite support sampling 
kernels such as the sine and Gaussian. 

A. Sine sampling kernel 

Consider a continuous-time signal with a finite number of weighted Diracs 

and an infinite length sine sampling kernel, see Figure 7. The sample values are obtained by 
filtering the signal with a sinc(j/T) , t G R, sampling kerneL This is equivalent to taking the 
inner product between the signal and a shifted version of the sine 

y n ^< x(t), sinc(t/T - n) > (74) " ' 

where sinc(i) — sm{wt)firL The question that arises is: How many of these samples do 
we need to recover the signal? The signal has 2K degrees of freedom, K from the weights 
and K from the locations of the Diracs and thus N samples, N > 2K % will be sufficient 
to recover the signal.. Similar to the previous cases, the reconstruction method will require 
solving two systems of linear equations: one for the locations of the Diracs and the second 
for the weights of the Diracs. These systems admit solutions if the following conditions are 
satisfied: 
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[CI j Raak(V) < if -f l where tw = A* ((-l)*nS») aadVe 
[C2 j Rank(A) * if where - g^g^ and A e R*** . 

Theorem 3: Given a finite stream of if weighted Diracs and a 3inc sampling kernel 
sinc(«/T). If conditions [Ol] and [C2] are satisfied then N samples with N > 2K 

y rt « < x(t),BiD.c(t/T - n) > ' (75) 

are a sufficient, representation of the signal. 

Proof: Taking the inner products between tbe signal and shifted versions of the sine 
sampling kernel yields a set of N samples 

y n = < 2<t), sinc(f/T -n)>, n = 0, . . . , N - 1 (76) 
Y^c k 6{t~t h ) 8 inc{tfT-n)dt (77) 

-oo «-» 
K-X 

= 52c fc sinc(ffc/r-n) (78) 
^ c A sic (?rtfc/r - nn) 

= ~ g <*sin (^/T) • ; . (81) 

The denominator of the previous expression (81) can be rewritten as follows: 



. (i k /T-n) ~ itf-'itu'T-n) P(n) 
where P{u) is a polynomial of degree if with zeros at all values of tfc/T, 



(82) 
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and the Pfe(tt) is a polynomial of degree K — l and has zeros at ail locations except at location 

Therefore if the coefficients of the polynomial P(«) are determined thett the locations of the 
Diracs are simply the K roots of P(u). We can now find am equivalent expression to (81) in 
terms of the interpolating polynomials: 

(-l) n P(n)S- « -^c fc sin (*WT)ft(n). (88) 

Note that the right-hand side of (85) is a polynomial of degree K - 1 in the variable n, 
applying AT finite differences makes the left-hand side vanish, 6 that is, 

A K ((-!)" P(n) fe) - 0, n - X, . . . >N - 1 (86) 
r«P*A K ((-l) n ^»n) -0 (87) 

V«p -0 (88) 

where the matrix V is an (N - K) x (K + 1) matrix and admits a solution when N — K>K 
and the rank(V) is less than K + 1, that is, condition [Oil- Therefore (87) can be used 
to find, up to a normalization, the K + 1 unknowns p* which lead to the K locations t k . 
Once the K locations t k are determined the weights of the Diracs axe found by solving 

the system in (81) for n = 0 K - L Since i k £ i u V* ^ I, the system admits a solution 

from condition [C2], * 
Note that the result does not depend on T. In practice if T is not chosen appropriately 
then the matrices V may be ill-conditioned. Figure 8(a) illustrates the conditioning of the 
matrix V is the least for T close to 0.5 and that the matrix A is well-conditioned on average, 
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By choosing more adequately the interpolating polynomials, for example by taking the La- 
grange polynomials, we may reduce the conditioning of the matrix V, but this remains to 
be investigated. The algorithm is as follows: 

Algorithm t: Finite length stream of Diracs sampled with a sine sampling kernel 

Given y n =< x{t),$inc{t/T - n) >, n ~ 0, 1, . . . , N - 1; 

Calculate v Hk = n h jfc), n = K, . . . , N - 1, k = 0, . > . , K; 

Solve the linear system V • p = 0 — > {Po,pi, . , Pa }; 

Find the if roots of * S>* ti* — * {V*\ * t /T\ . . . f ^/F}; 



Calculate K n ^ (-i)* p(«) y m n = 0, i, . . . , N - 1; 

Solve the linear system A • c =s Y — ^ {t^, c iT . „ . , cjr-t}. 
This method can be extended to piecewise polynomials, similarly to Theorem. 2. Also, there 
is an obvious equivalent for discrete-time signals in ^(&) and discrete-time sine kernels. 

B. Gaussian sampling kernel 

Consider sampling the same signal as in (73) but this time with & Gaussian sampling 
kernel, tp^t) = e~ f2 ^ 2 , see Figure 9. Similar to the sine sampling kernel, the samples 
are obtained by filtering the signal with a Gaussian kernel. Since there are 2K unknown 
variables we show next that N samples with N > 2K are sufficient to represent the signal. 

Theorem 4 : Given a finite stream of K weighted Diracs and a Gaussian sampling kernel 
Vtr{t) = eT e ^. If A r > 2K then the N sample values 



6 Note that the K finite difference operator plays the same role as the annihilating filter in the previous chapter. 



Calculate 




(89) 



March 23, 2001— 3 : 33 pm 



DRAFT 



SAMPLING SIGNALS WITH FINITE RATE OF INNOVATION 25 

are sufficient to reconstruct the signal, 

Proof: The sampte values are given by 



y n « < op(t), e-(^-n)W >, * = 0, . . . , JV - 1 (90) 
^i(f-4)e-(^(a (91) 



/ 



-co 

= £c fce ^^ 2 /^. (92) 

We expand (92) and regroup the terms so as to have variables that depend solely on n and 
solely on k. We obtain 

Wn = £(c,e-^H'^^-e-^ (93) 

which is equivalent to 

fc-i 

where we let y n ~ e" 2/2<r * y n > a k = e-**/ 2or2T2 and u k = e ih/o%T . Note that we reduced the 
expression 1^ to a linear combination of real exponentials. This hints that the annihilating 
filter method described in the Section III-A seems appropriate to find the K values u*. Let 
H(z) — ho + hi z~ x H h h K z~ H be an annihilating filter, that is, h is such that 

h * Y =0 (95) 
^ E^feK^ =0, » — J5f,._ f iST — 1. (96) 

Note that this is a Toeplitz system with real exponential components Y n = e* 2 ' 2 ** ^ acc j 
therefore a solution exists when the number of equations is greater than the number of 
unknowns, that is 7 N — K > K and the rank of the system is less than K + 1 which is 
the case by hypothesis. Furthermore cr must be carefully chosen otherwise the system is 

March 23, 2001—3 : 38 pm DRAFT 



SAMPLING SIGNALS WITH FINITE RATE OF INNOVATION 27 

ill-conditioned. If we factor H{z) — H i 1 ~ *r l u*) th en we obtain the locations of the 
Diracs tk from the roots of the polynomial H(z) i that is, 

h =a*T\t\u k . (97) 

Once the values of the Diracs t k are obtained then we solve for a k the Vatidermonde system 
in (94) for which a solution exists since u& ^ m,Vk 4" The weights of the Diracs are simply 
given by 

c h ^a h S^\ (98) 

■ 

Here unlike in the sine case, we have an almost local reconstruction because of the 
exponential decay of the Gaussian sampling kernel which brings us to the next topic, 

VI- Infinite length signals with finite local rate of innovation 

In this section we consider the dual problem of Sec. V, that is, infinite length signals 
x{t), t € with a finite local rate of innovation and sampling kernels with compact support. 
In particular, the /?— splines of different degree d are considered [8] 

<P&) = * <Po) (t) , deN~ (99) 

where <pu{p) is the box spline defined by 

, x f 1 if 0<f < 1 , . 

= { , , - ( i0 °) 

I 0 else 

We deveiop local reconstruction algorithms which depend on moving intervals equal to the 
size of the support of the sampling kernel. 7 The advantage of local reconstruction algorithms 
is that their complexity does not depend on the length of the signal. We begin by considering 
bilevel signals, followed by piecewise polynomial signals- 

7 The size of the support of ^d{t/T) is equal to (d + l)2\ 
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28 



A. Bilevel signals 

Consider an infinite length continuous-time signal x(t) 9 t G ft + which takes on two values, 
0 and 1, with initial condition a^*))^ = 1 with a finite local irate of innovation, p. These 
are called bilevel signals and axe completely represented by their transition values t k , For 
example, binary signals such as amplitude or position modulated pulses or PAM, PPM 
signals [3], see Fig. 10. 

Suppose a bilevel signal is sampled with a box spline tp^(tJT), Then the sample values are 
given by the inner products between the bilevel signal and the box function, 



It can bee seen in Fig- 11 that the sample value y n corresponds to the area, occupied by the 
signal in the interval [nT } (ft 4- 1)T]. Thus if there is at most one transition per box then we 
can recover the transition from the sample. This leads us to 

Proposition 3: A bilevel signal x{t),t > 0, with initial condition £(*)|fc-o — 1? is uniquely 
determined from the samples y n =< ar(t), <pn{tfT—n) > where '<p${t) is the box spline defined 
in (100) if and only if there is at most one transition t k in each interval [wT, (n 4- 1)T]. 

Proof: For simplicity let T — 1. Consider an interval [n, n 4-1] and suppose x{ri) = 1. 
First we show sufficiency followed by necessity . 

^ : If there are 0 transitions in the interval [n f n 4 l] then the area under the bilevel 
signal, or the sample value, is y n = 1 since we supposed that x(t)| tsssn> = 1- H there is 
one transition in [n> n 4 I] then the sample value is equal to 




(101) 




(102) 



— OO 




(103) 



n 



n 
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This implies that tk ~ Vn ~ n* Similarly if x(n) = 0 then we have tu = n-fl-j/ rt . 
Therefore we can uniquely determine the signal in the Interval [n, n 4- 1]. 
=>: Necessity is shown by couateremtnpk. 

Suppose #(n) = 1 and there are two transitions t ki tk+t in the interval [ft, n + lj then 
the sample value is equal to 

71+1 tfc n-M 

Vn « y y idf+ y (104) 

=±* fe-n + n + l-ttfi^fc-tfc+i + l. (105) 

That is, there is one equation with two unknowns and therefore insufficient samples to 
determine both transitions. Titus there must be at most one transition in an interval 
[ti, n + 11 to uniquely define the signal. 

■ 

Nov/ consider shifting the bilevel signal by an unknown shift c, see Fig. 12, then there 
will be two transitions in an interval of length T and one box function will not be sufficient 
to recover the transitions. Suppose we double the sampling rate, then the support of the 
box sampling kernel is doubled and we have two sample values y ni y n +i covering the interval 
[n + 1}T] but these values are identical (see their areas). Therefore increasing the 
sampling rate is still insufficient. 

This brings us to consider a sampling kernel not only with a larger support but with added 
information. For example, the hat spline function <Pi(t/T) defined by 

' 1 - |*| if \t\ < 1 



(106) 
0 else 



leads to sample values defined by y n =< x(t), <Pi{t/T — n) > or 

nT (n+lJT 

Vn - y *(*)(l + e/T-n)*+ y *(0(l-(*/T-n))dt. (107) 
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Prom Fig. 13 we can see that there are two sample values covering the interval [nT, (n+ 1)3"*],. 
We will show next that in this case we can uniquely determine the signal. 

Proposition ^: An infinite length bilevel signal with initial condition x{0) as 1 is 
uniquely determined from the samples defined by 

y n £± < x{t)>tp x {tJT ~ n) > (108) 

where ^>i(t) is the hat sampling kernel if and only if there are at most two transitions t k ^ tj 
in each interval [nT, (n + 2)7*]. 

Pwof: Again, for simplicity let T = 1 and suppose the signal is known for t < n and 

First we show sufficiency by showing the existence and uniqueness of a solution. Then we 

show necessity by a counterexample. 

Similar to the box sampling kernel the sample values will depend on the configura- 
tion of the transitions in the interval [n,n 4-2]. If there are at most 2 transitions in 
the interval [n, n + 2] then the possible configurations are 

(0,0), (0,1), (0,2), (l f 0) f (1,1), (2,0) 

where the first and second component indicate the number of transitions in the in- 
tervals [ira, n + 1], [n -f 1, n 4- 2] respectively, see Fig. 14. Furthermore since the hat 
sampling kernel is of degree one we obtain for each configuration a quadratic system 
of equations with variables £o>*i- 

y n = f x(t)(l + t-n)dt + f z{t)(l-{t-n))dt (109) 

n-1 n 
n+1 »+2 

lfc.+x = j* x{t){l+t-(n + l))dt + y *(t)(l-(i -(» + !)))#. 



(110) 
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First we show that the quadratic system of equations admits a solution and then that 
it is unique, 
(a) Existence. 

Take n == 0 and go the moving interval is [0,2]. 

The configuration (0, 0) will lead to sample values y 0 «* 1> Vi = 1- 

The configuration (0, 1) will lead to sample values 

*fc =*l/2 + ?(t-l)«tt ^f 0 2 -hl-*a (HI) 
i * 

|/i == 1/2 4- /(2 - t) dt = -ifc 3 * - 1 + 2t 0 (112) 

=*► £o = £/o + m = 1 + 7-1 -f- 2#o = 2 - ^/2~- 
The configuration (0, 2) will lead to sample values 

l/o ^o 2 + l-fc + tt~i*i* (H3) 
Vi = -^o 2 + l + 2^+^t 2 ~2< i (114) 

The configuration (1,0) will lead to sample values 

1 



2/1 - ^q 2 (116) 

to = Sfo + &i — 1 - \/I - 2y 0 = V^?!' 
The configuration (1,1) will lead to sample values 

Vo - -5<o a + to-^*i 8 +*i ( 117 ) 

m - \tf + 2 + \tf~2ii (118) 
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^ r 0 — — 2 1 f i ~ — ^ 

The configuration (2 7 0) will lead to sample values 

l/o « -~*o 2 + l + *G + gii 2 -<i (119) 
Vi « jfa' + l-jti* (120) 

=^ *o it-a+*i+ito) — >*i 3 <-a+w+*5 * 

(b) Uniqueness, 

If # n =s 1 and i/^i = 1 then this implies configuration (G, 0) . 

If y n = 1 and 1/2 < # n +i < 1 then the possible configurations ate (0, 1), (0,2). 

By hypothesis, there are at most two transitions in the interval [n + 1<« + 3] 

therefore if < 1/2 fchea the configuration in the interval [n, n + 2] is (0 3 1} 

otherwise if y n +i > 1/2 then the configuration i& (0,2). 

If 1/2 < tf w < 1 aiid 1/2 < < 1 then this implies configuration (2,0). 

If 1/2 < y n < 1 and 0 < y n +\ < 1/2 then this implies configuration (1,0). 

Necessity is shown by counterexample. 
Consider a bilevel signal with three transitions in the interval [0, 2] but with all three 
in the interval {Q ? I], see Fig. 15, Then the sample values in this case are are equal to 

y 0 = 1/2 + f{l-t)dt+ f (l-t)dt (121) 

0 ti 

= 1/2 + to -*i + <2- 4/2+^/2-^/2 (122) 
to ta 

Vl =s jtdt + J tdt (123) 

- *3/2-*?/2 + *J/2. (124) 

There is no unique solution for this quadratic system of equations. Therefore there 
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must be at most 2 transitions in an interval [0,2]. 

■ 

Once again if there is an unknown shift in the bilevel signal then there may be three 
transitions in an interval [nT, (ti 4- l)Tj and so we increase the number of samples by sam- 
pling: with v>i(t/(T/2))- The pseudo-code for sampling bilevel signals using the box and hat 
functions are given in full detail in [5J. 

When going to higher order splines, necessity carries over. Sufficiency is more tedious since 
we must solve a system of higher order polynomial equations. 

B> Piecewiae polynomials 

Similar to bilevel signals we consider sampling piecewise polynomials with the box sam- 
pling kernel. Consider an infinite length piecewise polynomial signal x{t) where each piece 
is a polynomial of degree R and defined on an interval [t^u f*L that is; 

m— 0 

: • (125) 



x(t) 



Each polynomial piece contains R+ 1 unknown coefficients c* m . The transition value tfc 
is easily obtained once the pieces x k -i(t) and x h {t) are determined, thus there are 2(R+1) + 1 
degrees of freedom. If there is one transition in an interval of length T the maximal local 
rate of innovation is p m (T) = (2(12 + 1) + 1)/T. Therefore in order to recover the polynomial 
pieces and the transition we need to have at least 2{R 4- 1) 4- 1 samples per interval T. This 
is achieved by sampling with the following box sampling kernel yo(*/ 2 (fl+i)+i )» For example 
if x(t) is a piecewise linear signal with 2 pieces as illustrated in Fig. 16 then to recover the 
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signal it is sufficient to take 5 samples: two before the transition, two after the transition 
and one sample coveting the transition* 

We can generalize by noting that the j?th derivative of a pieeewise polynomial of degree i? 
is a pieeewise constant signal. The pseudo-code for sampling pieeewise constant signals With 
the box sampling kernel is found in [5] . 

VI L Applications 

The applications we consider involve the discrete-time periodic stream of t)iracs and 
pieeewise polynomial signals. It is well known that a bandlimited signal can be perfectly 
recovered froin its samples by sampling it at twice the maximum frequency. What if the 
bandlimited signal has a jump or a discontinuity then the signal is no longer bandlimited 
and the usual method is not valid. These are what we call pieeewise bandlimited signals. 
Another type of non-bandlimited signal which we may come across in nature is a signal which 
is obtained from a system with a certain frequency response. The output of the system is a 
filtered signal. We will look at filtered stream of Diracs and filtered pieeewise polynomials. 

A. Pieeewise bandlimited signals 

A discrete-time periodic pieeewise bandlimited signal is the sum of a bandlimited signal 
with a stream of Diracs in the simplest case or with a pieeewise polynomial signal. An. 
example is illustrated in Figure 17(e) and is obviously not bandlimited from Figure 17(f). 
Formally, we have the following 

Definition 4- Pieeewise bandlimited signals. 
Let x^i be a discrete-time periodic L— bandlimited signal of period N with corresponding 
DTFS coefficients X. BL such that X BL [m] = 0 Vro 0 [— L,L]. Let x FP be a zero mean 
discrete-time pieeewise polynomial signal of period N with K pieces and with each piece of 
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maximum degree R. Then a piece wise handUmibed signal x is defined by 



2t =s X.BL + Xpp 



(126) 




(127) 



First consider a stream of K weighted Dirac^ xpp. From Section III- A, we can recover the 
K weighted Diracs from 2K contiguous frequency values Xpp. Since the DTFS coefficients 
of the bandlinuted signal, Xbl* equal to zero outside of the band [— L t L] t we have that 
the DTFS coefficients of the signal outaide of the b&nd L\ are exactly equal to the DTFS 
coefficients of the piecewise polynomial, that is, X[m] — Xpp[m], V|mj > L. Therefore it is 
sufficient to take the IK DTFS coefficients of the outside of the band [— L,L]> fot instance in 
[L+l, L-¥2K\ Suppose we have the DTFS of the signal X[rn], with m € [-(L+2K),L+2K) 
then the DTFS of the bandlimited signal are obtained by subtracting Xpp[m] from X[rn] 
for m e [— £, L]. 

Recall that the piecewise polynomial has 2K(R + 1) degrees of freedom and the bandlimited 
signal has 2L + 1. It follows that we can sample the signal using a discrete-time periodized 
differentiated sine sampling kernel bandlimited to 2K{R + 1) + L. 

Corollary 1: Consider a piecewise bandlimited signal x as defined in Definition. 4. Let 
il?[n] be the (R + l)th differentiated sine sampling kernel with DTFS 



Let M be an integer divisor of N, and let NfM > 2{2K{R + 1) + L) then the samples 



(128) 



feffl =< *[»], - IM\ >, I = 0, . . . , N/M - 1 



(129) 



are a sufficient representation of x. 
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Proof: The proof is exactly the same as in Theorem 1 until equation (43) 

- ~{l-W^Y^Mkhk^^.^aK{R+l)^L (130) 



_ f ^(1 - (X BL [k] + A>H*]) ifJfe^O,...,!, 

1 id - W^*)* 4 " 1 if Jb = £ + 1 2K{R + 1) + if } 

Therefore 2A"(i? + 1) values of 

*fj>M«* ^— ^ fc€[Ji + l J 2^(ie+.l)+£] (132) 

are sufficient to recover the piecewise polynomial X PP . From these we can recover the L 
spectra! components of the bandiimited signal since 

X BL [k) - ^—A^—{Y 9 [k\ - X PP [k)% h ^ 0, . . . , L. (133) 

This gives us the the baudlimited signal xbl and thus the piecewise bandiimited eignal as 
defined in Definition. 4 is recovered x ~ tc bl ±x. pp * ■ 
Figure 18 the illustrates the reconstruction of a bandiimited plus a piecewise constant 
signal using the following reconstruction scheme; 

Algorithm 2: Reconstruction of piecewise bandiimited signals. 
Require: N, M s N/M > 2{2K{R + 1) + L) + 1; 

Calculate- the samples y a [l] —< x[n\, 0[n - IM] >, I = 0,. , N/M — 1; 

Calculate the DTFS Jf[m], m € \-(2K(R + 1) + L), (2#(J2 + 1) 4- L)] from the DTFS of 

samples y a {t\ — > X PP {m] = X[m],m 6 [L + l s (2if(i? + 1) + X)]; 

Solve h * X P p[m] ^0 ? m6[L+l, (2K(# + 1) + I)] — > x PP ; 

Calculate X B l[^] = -X^m] — Xpp[ro], m € [— £, L] — ► x^ L ; 

The reconstruction is x = xbz, +Xpp» 
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£?. Filtered piecewise polynomials 

Another application of sampling piecewise polynomial signals consists in sampling their 
filtered output. Figure 19 illustrates that a filtered stream of Diracs is not bandlimited. 
These signals ate formally defined in the following 

Definition 5: Filtered piecewise polynomials. 
Let xpf> be a 2ero mean discrete-time periodic piece wise polynomial signal of period N with 
K pieces of maximum degree Jfc Let g be a filter with DTFS G. Then a filtered piecewise 
polynomial is defined by 

x — g * Xpp (134) 

and the corresponding DTFS coefficients X are defined by 

X[tn) » G[m] • Xpp{ml rn * 0, . . . , N - 1. (136) 
Suppose xj>p is a stream of if Diracs. If the filter has 2K contiguous nonzero frequency values 
G[m] then 2K frequency values of the signal X[rn] will be enough to determine 2K frequency 
values of the stream of Diracs, since Xpp[m] = X[m]/G[ln], and from Proposition 1 these 
are sufficient to recover the stream of Diracs . 

Corollary 2: Consider a filtered piecewise polynomial signal x as defined in Definition* 5 
with G[m] ^ 0 9 m £ [-K(R 4- 1) , K{R+ 1)]. Consider an (#4-1) differentiated sine sampling 
kernel ifi[n] with DTFS 

*[ro] - (D[m]) w Rect^ K{R ^ K{& ^ l)V (136) 

Let M be an integer divisor of .V such that NJM > 2K{R + 1) + 1. Then the filtered 
piecewise polynomial signal can be recovered from the N/M samples 

V$] =< v[n]>$[™ -IM]>> I ~ 0, . . . ,N/M - 1. (137) 
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Proof: Similar to the proof of piecewise bandlimited signals, we have that the DTFS 
coefficients of the samples y s \l] are equal to 

KM = jg{l-Wz?)™X[kl kz[-K(R + l) t K{R + l)) (138) 
AT 



= j7^~^ k )^{0\k}X PP [k}). (139) 



Since G[k\ # 0 for k € [~K{R+ 1), K{R + 1)1 we have 2K{R + 1) values of the DTPS of the 
piecewise polynomial 

X PP [k] - (1 _ w ^ )R+l G[k] fcffl, k e {-K(R + 1), K(R + 1)] (140) 

which are sufficient to recover Xpp and which leads to the filtered signal by Definition. 5. ■ 
The reconstruction scheme is described in the following algorithm and an example of 
the reconstruction is illustrated in figure 20, 

Algorithm 3: Require* N, Af, N/M > 2K(R + 1) + 1; 
Calculate the samples y 8 [l] =< x[n], p[n — IM] >, I = 0, . . / , N/M — 1; 
Calculate Y, - DFT^ /M ■ y s — > X{m), m € [-K{R + 1), JFC(A + 1)]; 
Calculate X PP [m] - X[rn}/G[m},m € [-A%R + 1), 1}]; 
Solve li * .X> F [ro] = 0, m £ -f l),K{R + 1)] — > xpp; 

The reconstruction isx = g*xpp. 
We have seen that the crux of the proof relies on the fact that the filter is known and is 
invertible over the number of degrees of freedom of the problem. What if the filter has a finite 
rate of innovation but is unknown? This is more complex and remains to be investigated* 
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3 x m « ta "i» " i« » j» a to -» ~« t » «■ 'sa 

(e) ro 

Fig. 3. (a) Periodic discrete-time signal with, if = 15 weighted Diracs with period A r ~ 256;(b) DTFS X[m}\ 
(c) Discrete-time periodized sine sampling kernel, tp[n]\ (d) DTFS Rect[~ KyI c\, K — 15(e) Sample values 
y a [f\ =< x[n] t ip[u - IM] >, I « 0, . . . , SI with M = 8; (f) DTFS Y,. 
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-»0 6l«Maa^<5"58M?2 



Fig. 4. Stream of jK" = 16 bunched Diracs with period N = 64. 




i j 4 s <$ t i f n ;i a » « is it 



Fig. 5. Comparison between the root finding method and the spectral extrapolation method on a signal of 
length iV = 64 v K varying between 2 and 16 on interval 2K< 100 simulations. 
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Fig. 6. (a) Discrete- time periodic pieeewise linear {R = 1) signal of period N ~ 1024 with K = 6 pieces; 
(l>) Differentiated sine sampling kernel,^ \n) = dfn) * d[n] * ip[n] with DTPS D[m\ • Rect[-. K ^ H ^i) > fc (it+i)] (c) 
Sample values ^[J] =< rfn], t% - f M] >, / = 0, . . . , 31 with M = 32^ (d) Stream of K(R + 1) = 12 Diracs 
obtained from X[mj,m € {-k(R + 1), JFf(H+ 1)]. 
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(a) (b) 

Fig. 6. (a) Average condition number of the matrix that leads to the locations of the Diracs, V, versus the 
sampling interval T, optimal T ft* 0.5; (b) Average condition number of the matrix that leads to the weights 
of the Dhacs, A, versus the sampling interval T s optimal T » L Average is taken on 100 signals with 8 
DiracB uniformly spread in the interval [0, 8]. 
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(a) 

Fig. 9. (a) Example of a finite length, continuous-time stream of K — S Diracs randomly spread on an 
interval [0, r] with r *± S; (b) Gaussian sampling k&rnei, f£v(£) = e~* ^,er ~ 2. 
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Fig. 10. Biievel signal. 
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Fig. 11. (a) Box spline sampling kernel, ^o(t), T = i. (b) Biievel signal sampled with the box sampling 
kernel. 
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Fig. 12. Shifted bitevel signal with two transitions in the interval [5,61. 
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(b) 



Fig. 13. (a) Hat spline sampling kernel,(/?(t/r), T = 1. (b) Bilevel signal with two transitions in an interval 
[n, n + 1] sampled with a hat sampling kernel. 
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Fig. 14. Bilevel signal containing at most 2 transitions in the interval [0,2]: All possible configurations. 
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Fig. 15. Bilevel signal containing three transitions in an interval [4. 5] ? sampled with the hat sampling kernel 

nit). 
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Fig. 16. Piecewise linear signal sampled with a box sampling kernel. 
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Fig. 17. (a) Baadliraited signal of length N - 256; (b) DTFS of Baadlimited signal, L = 15 (c) Fiecewise 
constant signal with K — 3 pieces; (d) DTFS of piecewise constant signal; (e) Baiidiiniited piecewise constant 
signal; (f) DTFS of bandiimited piecewise constant signaL 
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(c) (d) 

Fig. 18. (a) Bandlimited piecewiae conatant signai,a; [n], with K = 3, L — 15,il = 0,iV = 256; (b) 
Differentiated sine sampling kernel, ip\n] = d{n] * v?[n], bandiimited to 2K(R + 1} + 1 + L — 22 (c) Sample 
values «[n],^[n-IJ<l >,i = 0,... ,iV/M-l, Af - 4; (d) IZXTjFSI of sample values; Reconstruction 

error is 10 -13 . 
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(e) (f) 

Fig. 19. (a) Stream of K - 4 Diracs with period N = 64; (b) \DTFS\ of stream of Diracs (c) Known filter 
g[n) = a n ,n = 0, , . . , AT - 1, <m ^ 0.4; (d) jJ2TFS| of filter; (e) Filtered stream of Diracs; (f) \DTFS\ of 
filtered stream of Diracs. 
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(c) (d) 

Pig. 20. (a) Filtered stream of Diraca, x[n], N = 64; (b) Sine sampling kernel v?[n] b&ndlhmted to X = 4 (c) 
Sample values y 8 [l] ~< xln]*<p[n-lM] >,/ = 0, . , 15, Af ~ 4; (d) jDTFS) of sample values; Reconstruction 
error is 10~ 13 . 
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VIIL Conclusion 

• We derived sampling theorems for periodic signals in particular streams of weighted 
Diracs and piecewise polynomials, These signals have a finite rate of innovation p 
Which is equal to the number of degrees of freedom per period- 

• The samples are obtained by taking the inner product of the signal with a shifted 
version of the periodiased sine kernel or differentiated sine kernels. The bandwidth of 
these kernels must be greater or equal to the degrees of freedom of the signal. 

• The discrete-time periodic signals are perfectly recovered when the sampling rate 
l/M is greater or equal to the rate of innovation p 2KJN in the case of streams 
of weighted Diracs or p « 2K{R + l)/N in the case of a piecewise polynomial signal 
with K pieces and maximum degree R, 

• The continuous-time periodic streams of Diracs and piecewise polynomial signals are 
perfectly recovered when the sampling rate 1/T is greater or equal to the rate of 
innovation p = 2K/r since we assumed that the piecewise polynomial signal belonged 

• The sampling and reconstruction scheme is illustrated in Figure 21. 
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Fig. 21. Sampling and reconstruction scheme for discrete- time piecewise polynomial signals with K pieces 
of maximum degree R; JY/Af Is the number of samples*, 2K(R 4* 1) + 1 is the bandwidth of the sampling 
kernel* 
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• A finite length stream of K Diracs can be recovered from N samples ttn obtained as 
the inner product between the signal and shifted versions of the sine and Gaussian 
sampling kernel, when AT > 2K. 

• For both types of sampling kernels two systems of equations must be solved; the first 
system is to find the location^ of the Diracs and the second is to find the weights of 
the Diracs. 

» When sampling a randomly spaced stream of Diracs with the sine kernel the system 
leading to the transitions m&y be ill-conditioned if the sampling interval T is not 
chosen appropriately. It is illustrated that at critical sampling, that is 3 when we have 
N =± 2$C sample values, the optimal sampling interval obtained fot these type of 
signals is T — 0.5. 

• When sampling a randomly spread stream of Diracs with the Gaussian kernel the 
conditioning of both systems depends also on the value of the variance in the 
Gaussian kernel. 

• The sampling schemes using the sine and the Gaussian kernels can be generalized to 
both continuous-time and discrete-time piecewise polynomial signals. 

• Infinite length signals were sampled using a compact support sampling kernel. 

• Bilevel signals can be recovered using a Box sampling kernel ip^{t(T) if and only if 
there is at most one transition in each interval \n y {n + 1)T). 

• Bilevel signals can be recovered using a Hut sampling kernel ip x {ijT) if and only if 
there is at most two transitions in each interval [ra, (n + 2)T], 

• In general, to recover the infinite length piecewise polynomials with K pieces of of 
maximum degree R using a box sampling kemeij the sampling rate must be greater 
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than the maximum local rate of innovation 

Ab (T)«(2(1I -M) + l)/T. 

• Sampling and reconstruction algorithms were given for each problem in theit respective 
sections. 
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Appendix 
I, Annihilating filter method 
The problem in spectral title analysis consists in estimating the frequencies of a sinusoidal 
signal from a set of values. The methods used for estimating such frequencies are known as 
high-resolution methods, for example MUSIC, ESPRIT and can be found in [7|. We define 
the following as the annihilating filter method: 

Consider a signal ^[n], n € % defined as a finite linear combination of K exponential u% 

K-l 

*M - ^2^kUt (141) 

where c* are real and u k are real or complex valued. In the context of spectral line analysis 
u k ~ e*^ where w k is the kth frequency component of the signal s[n]. 

Definition 6: A filter 1 ~ z~ l u k is called an annihilating filter for u% if [7] 

(1 - z- l u k )ut = 0. (142) 
where z~ l is a shift or delay operator. 

Suppose that ^ 0, then z — u k satisfies (142) and is a zero of the filter 1 — z~ l If there 
are K exponentials then there are K filters 1 — z~ l u k each ajinihilating their respective u% 
This implies that the product of these filters 

H{z)^l[(l^z' l u i ) (143) 

for sure annihilates each exponentials, uj, thus H(z) is an annihilating filter of s{n], 

s[n] ■ H{z) =s 0. (144) 

Therefore to find the values u k we need to find the filter coefficients in 

K 

H(z)^J2 h * z ~ k (145) 
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such that (144) is satisfied. Substituting H{z) defined in (145) in (144) we obtain 

K 

52 h k s[n] z~ h = 0, n 6 % 
which equivalent to the following recurrence equation 



(146) 



h*s ~ 0. 



(147) 
(148) 



In matrix/ vector forih the system in (147) is eqtiivalent to 



s[0) s[~l] 



S [-ifT] 
•[0] 



/ h A 



= 0. 



(149) 



Suppose a finite number of values s[n] ate available. Since there are K -f 1 unknown filter 
coefficients we need at. least K + 1 equations, and therefore we need at least 2K + 1 values 
of a[n] to find the filter coefficients. 8 Define S the appropriate sub-matrix then the system 
S • h — 0 will admit a solution when 



ranktS) <K + l. 



(150) 



Once the the filter coefficients are found then the values u h are simply the roots of the 
annihilating filter H(z). 

To determine the weights c k it suffices to take K equations in (141) and solve the system for 

c fc . Let n = 0, . , . , K — 1 then in matrix vector form we have the following Vandermonde 

8 Actually there are K unknown niter coefficients since fto = 1 and therefore we will used at least 2K values of 
» [to]. The system to solve in this case is known as a Yule-Walker system [2] 
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system 



1 



1 



and has a unique Solution when 



1 



/ e 0 \ 



CO 



,[0] \ 
s[i] 



\ b[K-1] J 



(151) 



(152) 



This concludes the annihilating fitter method. 
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LtST of Figures 

1 Sampling set up: x(t) the coiitiiluaiis-time signal; (p(—t) is the sampling kernel; 
T is the sampling interval; $[n] —< x(t)> <f(t - nT) >, n = 0, . , . , N - 1 are the 
sample values 27 

2 (a) Periodic discrete-time signal with K = 8 weighted Diracs, x[n], N = 255; 
(b) Discrete-time perlodiaed sine sampling kernel, <p{n] with DTFS <Mm] == 
ttect[-MMh M — ®* ( c ) Sample values yt — < &[n],^[ft— £T| >, I =» 0, . . . ,31 
with T 8; (d) Reconstructed stream of Diracs, MSB - 10~ 13 . 28 

3 (a) Periodic digcrete-tirrie piecewise constant signal, ar[*i], of length N = 266 
with K' = 8 transitions; (b) Stream of if = S weighted Diracs obtained by 
applying the discfete-time difference operator to a;[n], d[n] * x[n\] (c) Discrete- 
time periodized sine sampling kernel, with DTFS $[tn] •= jRec^-kfjtfj), M = 
9; (d) JV/r Sample values ^ -< s[n], y[n - /T] >, { = 0, . . . , 31 with T = 8; 
(e) Reconstructed stream of Diracs, MSE = 10* l2 ; (f) Reconstructed piecewise 



constant signal, MSE = 10" 13 . 29 

4 (a) Finite length continuous-time signal with K = 4 weighted Diracs, x(£} ? £ € 
[O,?-], r = 8; (b) Continuous-time sine sampling kernel, <p(l) = s£ne(£) ; (c) 
Sample valuer y u ==< x(t), sinc(t/T — n) >,n = 0, . . . , ???; (d) Reconstructed 
finite length continuous-time stream of Diracs, MSE = 10"" ?? 30 

5 (a) Conditioning of t;^ n versus the sampling interval T; (b) Conditioning of 
Vandermonde matrix versus the sampling interval. 31 
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6 (a) Finite length continuous-time signal with K — 8 weighted Diracs, e 

[0,r|, r == 8; (b) Continuous-time Gaussian sampling kernel, <p(t) — ; (c) 

-(f T .o* 

Sample values =< s(t),e ^ >,n = 0 ( - , AT — 1; (d) Reconstructed 
finite letigth continuous-time stream of Diracs, MSB = 10~ rr 32 

7 (a) Conditioning; (b) Reconstruction error versus conditioning. 32 



8 (a) Btlevel signal with at most 1 transition in each interval [n, n 4- 1]; (b) Box 
sampling kernel, <poC0; £c) Sample values a;[n] x(i),^o(£ — *»T) > with 7" s= 1. 32 

9 (a) Biievel signal with at most 2 transitions in each interval [n t n 4- 2]; (b) Hat 
sampling kernel, <pi(t); (c) Sample values a;[n] — < ^(i), <pi(i — rtT) > with T === 1, 32 

10 Nou intersecting solution spaces for bileve! signals wtth h&t sampling kernel , . 32 

11 (a) Periodic discrete-time bandlimited signal of length AT — 1024 With 
Mql = 4; (b) Periodic discrete- time piecewise constant signal,xpvf ? with K =s 4 
transitions; (c) Periodic discrete- time piecewise constant bandlimited signal, x — 
xbl + with K = 4 and M^16; (d) Sample values yj =< £[nj, 0[n - KT] > 
,/ = 0, . . . ,N/T - I, T = 32 5 where V( n l ~ <p[n] - - l):(e) Reconstructed 
piecewise constant signal, MSB — 10~ ?? ; (f) Reconstructed piecewise constant 



bandlimited signal, MSB = 10~ ?? . . . . 32 

12 (a) Discrete-time stream of K = 4 Diracs, x^iy; (b) Known filter g[n] = a n u[n], 
a = 0.4 (c) Filtered stream of Diracs, x = g * xpwl (d) — < z[n] y <£[n — iT] > 
J = 0,,.. ,31, T = 32;(e) Reconstructed stream of Diracs, MSiS = 1(T™; (f) 
Reconstructed filtered stream of Diracs x, MSE = 10~ ?r 32 

13 General reconstruction scheme 32 
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Fig. I. Sampling set up: x{t) is the continuous-tinte signal: tp{-t) is the s&mpiiag kernel; T is the Sampling 
iiitervali x[n] x{t)^{t - nT) >, ti = 0, . . . , N - 1 ate the sample values. 
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Fig, 6. (a) Finite length continuous-tune signal with K = 8 weighted Diracs, € [0,r| t r = 3; (b) 

Continuous-time Gaussian sampling ketnei, <p(t) = eai£ ; (c) Sample values # n r-< > 4 n = 

0,- . . , AT - i; (d) IWonstructsd finite length contimiou9-titne stream of Diracs, USE — 10*". 

Pig. 7. (a) Conditioning; (b) Fterxmstructiori error versus conditioning* 

Fig, 8. (a) Bilevel signal with at most 1 transition hi each interval [n, n 4- 11; (b) Box sampling Fernet, wn(t\: 
(c) Sample values w[n] ar(t), 9o(* - «T) > with T = L ' W 

Fig. 9. (a) Bilevel Signal with at most 2 transitions in each interval [n.n -f 21; (b) Hat gambling kern«L 
<A (t); (c) Sample values x[n] =< ^(t - n T) > with T - 1. 

Fig. 10. Noti intersecting solution spaces for bilevel signals with hat sampling kernel 




(c) (d) 
Fig. 11. (a) Periodic discrete-time bandiimited signal x BZ/ of length N = 1024 with M Bh =*- 4; (h) 
Periodic discrete-time piecewise constant signal,x P jv ; with K = 4 transitions; (c) Periodic discrete-'time 
piecewise constant haadl united signal, x = x BL + x PW , with if = 4 and M=16; (d) Sample values t/* =< 
- HI >i* - 0, . . JV/T — 1, T = 32, where = p[ n ] - <p{n - l];(e) Reconstructed piecewise 
constant signal, USE = 10 (f) aeconstructed piecewise constant bandiimited signal, MS£ = i0~ 7? - 

Fig. 12. (a) Discrete-time atTeam of K = 4 Dixac&, x PW /; (b) Known filter s[n| = a"*Jn], a = 0 4 (c) 
Filtered Btam ofDixacs, x = g *X fw ; (d) » s{nU[n - IT] >, f = 0. . . . ,31, T = 32;(e) Reconstructed 
stream of Dnacs, MS# ~ 1Q~ (f) Reconstructed filtered stream of Diracs x, USE = 10~ ?? . 

Fig. 13. General reconstruction scheme 
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Sampling periodic signals with 
finite rate of innovation 

Sampling theory has been extenaively developed For bandliniited signals. In this 
chapter 1 non-bandlimited signals axe investigated in particular periodic signals 
with a finite rate of innovation. Recall that in Section Lt signals with a finite 
rate of innovation p are characterised by having a finite number of degrees of 
freedom per unit of time. For example take a periodic signal of period N with 
Diracs at K locations. This signal is not bandJmrited and lias K degrees of 
freedom in an interval of length N thus its rate of innovation is p = KjN. 
In Sections 2.1 and 2.2 of this chapter* sampling theorems for discrete-time and 
continuous-tune periodic streams of weighted Diracs and piecewise polynomial 
signals are derived. Both of these type of signals are not bandlimited and have 
a finite number of degrees of freedom per period. By taking an appropriate 
sampling kernel and a sufficiently high sampling rate that captures these de- 
grees of freedom, the signals can be perfectly reconstructed. Section 2.3 derives 
applications of the above results, in particular to piecewise bandlimited signals, 
and to filtered piecewise polynomials. In all of the proofs of the aforementioned 
sampling theorems a method, that is commonly used in spectral analysis is em- 
ployed, namely the "annihilating filter method" [67]. For those unfamiliar with 
this method it is described an Appendix 2. A. 

2.1 Discrete- time periodic signais 

The discrete-time periodic signals we consider are streams of weighted Diracs 
and piecewise polynomials. Through appropriate differentiation, piecewise poly- 
nomials can be reduced to streams of Diracs, so we begin with these. 

2.1-1 Stream of Diracs 

Consider a discrete-time periodic signal, with one period given by 

x - (40|^[l] 3 ...,x[A'-l]) T (2.1) 

T This chapter includes research conducted jointly with Martin Vetterli and Thierry Rlu 
[80, 79, 7&J. 
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and containing K weighted Dimes at Locations {tio,^, . . . , }, n k [0\ IV - 
l] and if < (tf/2j, 

Jc-i 

where S[n] is the Kronecker delta and equal to 1 if n =* 0 and 0 if n ^ 0- 
Denote by X - (X[0], . . . - i]f the discrete-time Fourier series 

(DTFS) coefficients of x where 

Ar-i 

X[m| - l>fctt^"\ m=0,...,JV-l (2.3) 
and « e -u*fN 

Consider filtering the signal z{n] with a lowpase filter <£[n] — ^[-rt] with band- 
width [~~K,K\ then the sample values yJl] are simply a siibsampled version (by 
A£) of the filtered signal y[n] - &[n\ * (p[n\. The DTFS coefficients of pin) are 
given by 

and those of the subsarapled signal y 8 [l] = y[lM] are given by the usual sub- 
eainpling formula 

Af-i 

Y*M = ~ D + (2.5) 

With appropriate re-indexing it follows that 

y *M « Tj^M, m e [- JC a -fiH- (2.6) 

Figure 2.1 illustrates that we can recover 2K spectral values X [ml of the original 
signal from the subsampled spectra of the lowpass approximation l^[ru] as long 
as there is no overlapping in the spectra of the lowpass approximation Y*[m] and 
this occurs only if NfM > 2K. This leads us to 

Proposition 2,1 Consider a discrete-time periodic signal x\n] of period N 
containing K weighted Diracs. Let M be an integer divisor of N satisfying 
N/M > 2K 4- 1. Consider tte discrete-time periodized sine sampling kernel 
K 

p[n] ^ £ Wn™, that is, Hie inverse DTFS of the RecU~K t K\* Then the 
N/M 6 N samples defined by 

vAf\ = <x[nl<f[n-lM]> ciTC , I = 0, . . - ,N/M - 1 (2.7) 
are a sufficient representation of the signal 

Proof: We start by showing' that the DTFS coefficients X[m\, rn e [-K\ K] 
are sufficient to determine the stream of K weighted Diracs - Then we show that 
the N/M samples y ff [l] are a sufficient representation of X[m} y m e [— jK", K\. 



2.1. Discrete-time periodic signals 



I. Since }C[m] is a linear combination of iT complex exponent Jala, ug 1 , with 
Uk = flTpf* s the locations n* of the Diracg can be found using the annihi- 
lating filter method described in Appendix 2.A. It stiii&eeg to determine 
the annihilating filter H{z) whose coefficients are (1, !?[!], . . . , H[tC\) or 

if (*) = 1 + t£[l] 2T 1 + H{2\ z~ 2 + -" + R[K] z ~ K (2.8) 

which factors as 



jfe=0 



and satisfies 



(2-9) 



^ff[ft]X[m-fc] =±0, m=0,...,iV-l (2.10) 

Since #[0] =± 1, K □qn&tions (2.10) will be sufficient to determine the K 
unknown filter coefficients Hlk]^ = 1,..- ,rY. Let m = 1,... ,K then 
the system in (2.10) is equivalent to 



(2.11) 



For example take iY = 8, *T = 3 and let tn = 1,2*3 then in matrix/vector 
form the system is 



X[0) X[-i] X[-2\ 

x[i) x[o] xi-i) 

X[2] X[l) X[0] 



[ H[2] ) - - X[2] .(2.12) 



Given that these are K sinusoids the matrix in (2.12) is hill rank (— K) 
and thus there is a unique solution H[l\, . . . , H[K\. The set of locations 
{no, nu - • * ?^K-i} given by the the zeros of H(z). 
The weights of the Diracs axe obtained by solving K equations in (2-3), 
let m = 0 t . . . , K — 1, this leads to the following Vandermonde system 



1 



1 



X[l] 

\X[K-1] } 
(2,13) 



and has a unique solution since the n* ^ nj ? Vft ^ /. 
Therefore, given 2K contiguous DTFS coefficients 

{X[-K + i] , JT + 2], - . . , A-fDJ, . . - , X[K\} 

we have found a, unique set of locations {tia}^ 1 and a unique set of 
weights {c*}£r 0 l - 
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2. We need to show that 2iC spectral vstlues X[m\,m (= can be 

obtained from the N/M sample values y s [t[ defined in (2.7). 
We substitute the discrete-time periodized sine kernel ill the expression of 
She Sample values and we obtain the following: 

|T«[q = <^N 3 ^-IM]> CJ ^ ^0 3 ...,iV/M~l (2.14) 



= 22 *w ~ ^ 


(2.16) 




(216) 




(2.17) 




(2.18) 








(2.19) 



If we calculate the DTFS coefficients of the sample values y^[i\ we obtain 
an expression in terms of the DTFS of the signal* 

= v £ £ *H™W/^&m (2-21) 
f=0 m— —AT 

= ~ r *{-»*>] £ <^ (2-22) 



■I 



N[M if fc + rn = 0 
0 otherwise 



= ^A'fJb], k^0,...,nmi{2C,JV/M ~ 1} (2.23) 

= AfY»flb], fc = G,... 3 X (2.24) 

by hypothesis, iV/A4" > -t- 1 > K. Since we are dealing with real 
signals the DTFS is Hermitian, that is, X[-k] = X m [k], k = 0, . . . , /T, so 
we have the 2# 4> 1 spectral values X*^], k € [—iT, j£j obtained horn the 
JV/Af DTFS coefficients of the sample values Therefore we have & 

sufficient number of spectral values which unicuely define the stream of 
weighted Diracs. 
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Figure 2.2 illustrates b time and frequency domain the sampling of a discrete- 
time periodic stream of Diracs with period N = 256 eUid K == 15 weighted 
Diracs. The signal is perfectly reconstructed within machine precision* M 5fi 2= 

Note that iu the proof of Prop. 2.1 the locations of the Diracs are determined 
hy finding the roots of the aimihilating filter Biz). If the locations are bunched 
Up or there are a large number of Diracs then finding the roots of the poly- 
nomial is numerically unstable. An alternative method that is commonly used 
in error correction coding involves extrapolating the N - K spectral values of 
the signal using K first spectral X^k = 1, . f K components and the er- 
rot locating polynomial which in our case corresponds to the annihilating filter 
li- 

K 

*W « - H #M X[h - IJ f * - If + 1, . . . , .V - IT. (2.25) 

Consider a signal of* length N =± 64 where there are =s 16 Diracs in an interval 
of size 2K, see Figure 2.3. Figure 2.4 compares the relative reconstruction error 
between the root finding method and the spectral extrapolation method for 
different values of K. 

2.1.2 Piece wise polynomials of degree R 

The previous result on the stream of Diracs is extended to piecewise polynomials. 
Consider a discrete- time periodic piecewise polynomial defined by 2 




(2.26) 



of period N with K pieces each with maximum degree R. Suppose a discrete- 
time difference operator d[n] = ${n] — S[n — 1] is applied E + 1 times to the 
piecewise polynomial signal. The differentiated signal x 1 ** 1 ^] in frequency 
domain is 

Y<*+i)[ m ] - (Dim])"* 1 X[m], m = 0, . . . , N - 1 (2.27) 

where £>[rn] = 1 — Wp is the DTFS of the discrete-time difference operator. 
This results in putting to zero all the polynomial pieces. Assume there are 
discontinuities between pieces (but no Diracs), then K transitions can lead 
to at most K{H + 1) weighted Diracs and thus the rate of innovation is p = 
2K(R-r-l)/N, From Proposition 2.1 we can uniquely recover the Jf(i2-f 1) Diracs 
from 2K{R + 1) DTFS coefficients of the dL&'erentiated signal The 
piecewise polynomial signal is reconstructed by applying the inverse discrete- 
time difference operator R 4- 1 times on the stream of weighted Diracs. The 
discrete-time difference operator d{n] is a singular operator (since D\fi] = 0) 
and so we define the inverse discrete-time difference operator as D~ L [m] — 0 
for m = 0 and D~ 1 [m] = (1 - Wff)~* for m = 1, . . . a iV - 1. Hence instead 
of using the sine sampling kernel tp[n] we will use the derivative sine samphng 



'*n± = n, if n > 0, and 0 else. 
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kernel defined by = ( g( * *- ;_• * which has at least H + l zeros at 
the origin * « L Then the DTFS of ia 

$[mj =s (1 - rt^)* 4 " 1 *[ml, m = 0, . . . , H - 1 (2.28) 



where $[m] & the l?ee£[_ /<■ < jt+i), K(H+i )] function. This brings us to the following 
theorem. 



Therein 2.1 Consider a di&crete-iime periodic piecetirtse p&tptwrnial sigtiai of 
period N with K pieces Of degree R and atith zero mean. 3 Let M fa an integer 
and a divisor of N such that N}M > (2K(H 4- 1) + 1). Take a sampling kertid 
ip[n\ with DTPS coefficients defined in then me can recover the signal 

from the NJM € N samples 



= < xjrtj, ii>\n - I M\ > 0, . . . , N/M - 1. (2.29) 



Prooft First we show that + 1) DTfS coefficients of the signal, 

X[m) t m E \-K{R -r X),K{R + 1)] are sufficient to determine the piecewise 
polynomial aign&l, ar[«}_ Then we show that; the N/M samples y s \l) axe suffi- 
cient to determine the (2K(M+ 1) + 1) values X\m\, 



if we have the DTPS coefficients X[m],m € [-K{R 4 l),K{R+ 1)} then 
from (2.27) we have the DTFS coefficients of the (R + l)th discrete-time 
differentiated signal, X^^Jm]. From Prop. 2.1 these are sufficient to 
reconstruct the stream of K (R+i) Diracs. Thus, the signal ia recovered by 
applying #+1 times the inverse discrete-time difference operator, d~ l {n], 
on the stream of Diracs, that is, 



2, Similar to the second part in the proof of Prop. 2.1 we expand the inner 
product between the piecewise polynomial signal and the diiferentiated 



3 We consider zero mean siguaJs since Do D~' 1 tea projector on the 3pace t>f signal having 
zero mean. 
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sine sampling kernel: 

p 6 \i] « <*[n],^[rt-/JW]>, 1 = 0,... f JV/Af-l (2 JO} 

* »E*M £ (l-W^ 1 ^"- 1 ^ (2.32) 



Taking the DTFS of the sample values y,[J] we obtain 
jvyjwr-i 

tiffc] - 52 ^W^Afi fc = 0,.,.,^/M-l (2.36) 



- j? E E ^ m) (2.38) 

f JV/Af iffe + m^O 
~[ 0 otherwise 

= 77 (1 - ^ fe ) H+1 X[k), k = 0, . . . , min{N/M, X(H + 1)} 

(2.39) 

- I M[(i-w^*)«+»]-V.t*] fe&=i,...,jr(B+i) (240) 

Since iV/Af > (2£f(# 1) + 1) we have a sufficient representation for the 
spectral values of the signal. This completes the proof. 



Figure 2.5 illustrates the reconstruction of a, discrete-time periodic piece wise 
linear signal of period TV - 1024 with K ~ 6 pieces. 
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2.2 Continuous-time periodic signals 

We derive now the equivalent results but for continuous-time periodic signals, 
again building up from a stream of Diracs to piecewise polynomials. We wtfl 
put in evidence the common points. 

2.2.1 Stream df Diracs 

Consider a continuous-time periodic Signal t{t) of period t containing AT weighted 
Diracs at locations Ofe}j^ J with ifc € [0,r), or 

= c t i({-ft + ftT)) (2.41) 

4=0 

sirjce tn+tc = 4- f and c rt+ j< c« for all n £ N. 

The continuous-tune Fburier series (CTFS) coefficients of x(t) are defined by 

X[m] - ~fx(t)e- nirlrrl ! T dt, mZZ 
o 

i K'-l 

« - 52 ck e- <2 *^ m/r . (2.42) 

If the signal is convolved with a sine filter of bandwidth [—If, K] then we 
have a lowpase approximation yit) given by 

Suppose the lowpass approximation y{t) is sampled at multiples of T, we obtain 
t(T € N samples defined by 

*M = £ xHe^, / = o,... ,x/r- 1. (2,44) 

Similar to the discrete-time case as Long as the number of samples is larger 
than the number of values in the spectral support of the iowpass signal, that is, 
J > 2K + 1 5 (2.44) can be used to recover 2K 4- 1 values of X]pC[. Thus we can 
state: 

Proposition 2*2 Consider a continuous-time periodic stream of K weighted 
Diracs witft period r and a continuous-time periodic sine sampling kernel tp(t) 
with bandwidth [~K y K\. Taking a sampling period T such that r/T e N and 
r/T >2K + 1. Then the samples defined by 

= <x(t)Mt-tT)>> « = 0,... f T/r-l (2.45) 

are a sufficient representation ofx(t). 
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Proc/; Similar to the discrete-tittle rase first we show that 2KT -h 1 CTFS 
coefficients X [m\ are sufficient to find the locations and the weights of the blracs. 
From (2.42) we have that the CTFS coefficients X[m] are linear combinations 
of complex: exponentials. Thus to find the locations h we need to find the 
annihilating filter H = (L H [2], . , such that 

H* C X =a 0. (2.46) 
Th& is the game Toeplitst system as in (2.10) considered in Sec. 2.1.1 attd there- 
fore a solution exists. Factoring the z-transform of H, or H (z) = ^ 
into 

H{z) - Hil-z^Ut), (2.47) 
i=o 

we then find the K locations {f-o,tt,. . . from the zeros of H{z), that is, 

from 

Uk « e -«*W^ (2.48) 

Given the locations {tt}^ 1 and K values X [m], m = 0, . . . s #C - 1 4 we find the 
weights {ck}tf~v °^ ^ fcfracs by solving the Vandermonde system in (2.42). 
Since the locations t k are distinct, ^ tf,Vfe ^ i, the Vandermonde system 
admits a solution. 

The second part of the proof consists in showing that the r/T samples y s [(] are 
sufficient to determine the CTFS coefficients X[m]^fn e [~-K 9 K\. We substitute 
the continuous-time periodic sine function tp{t) with bandwidth [— fC, K\ defined 

by 

K 

in (2.45) and we obtain 

y s [l) = -IT) >, i=0,...,r/T-l (2,50) 

r ^ 

= X) e»*^'- |3r) ' T dfc (2.61) 



0 



(2.52) 



K 

= r ^ Xhr«]e^ ralT/f . (2.53) 



Note that * 3 periodic with period r/T, thus the DTFS coefficients are 
Y g [k] = TX [ft], Jfe = 0, . . . ,r/T - 1. Since r/T > 2if + 1, we have a sufficient 
number of samples that determine the CTFS X[m] 7 ro € [— /£, iC]. ■ 
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2.2,2 Piecewise polynomials of degree R 

Here we consider continuous-time periodic piece-wise polynomial signal of period 
r, containing it: pieces of ma^dmum degree R and ft - 1 continuous derivati ves. 

*M - 31 51 & ~ * € [0,rj. (2.54) 

We differentiate the signal H + i times and we obtain a continuous- time stream 
of K weighted Diracs, x^ HH ^{t). The CTFS of the derivative operator is dafined 
hy D[m] = t2#tn, tn e Z and thereto* the CTFS coefHcientS of the differentiated 
signed ar< J?+1 >(i) ate equal to 

X^Vl - (Mm)** 1 xNi » 6 (2.55) 

Btetttt Proposition 2.2 we can recover the continuous-time periodic stream of 
K Diracs from the CTFS coefficients, X^ n ^[m] 7 in € j-jfC,*^. Therefore we 
can sample the signal with the differentiated sine sampling kernel whose CTFS 
coefficients are denned by 

*[m] = {i2irm)^ 1 *[m} 9 me£ (2.56) 

where &[m] — Rect[„ KtK ] is the CTFS of the continuous-time periodized sine 
sampling kernel. 

Thedlrem 2-2 Cotisider a continuous- time periodic piecewise polynomial signal 
x(t) With period r, containing K pieces of maximum degree R, betoriging to 
gk-i an £ hwiftg zer0 m&jtfit Consider a sampling kernel t£(t) Unth its CTFS 
coefficients defined in (2.56). let rfT e N and tJT > 2K + 1, Thtn x(t) can 
be uniquely recovered from the t/T samples 

VB [f\ =< x(t)^{t - IT) >, I = 0, . . . , t/T - 1. (2.57) 

Proof: Similar to the proof of Theorem. 2.1 , we first show that CTFS coef- 
ficients X[m],m e [—K,K] are sufficient to determine the piecewise polynomial 
signal, x(t). Then we show that the t/T samples y t [l] are sufficient to determine 
the values J£[m],m 6 [— K,K\. 

1. If we have the CTFS coefficients X\m} ? m e [~K S K] then from (2.55) 
we have the CTFS coefficients of the [R + l)th differentiated signal, 
Xl R+1 )[m], From Prop. 2.2 these are sufficient to reconstruct the stream 
of K Diracs. Thus, the signal is recovered by integrating R+l times the 
stream of Diracs, that is, 

X (t) = J J ... J x( ii + 1 )(t)dt,dt-*dt 

or in frequency domain from (2.55) 

X{m] = p-Hm])^ 1 X^^lml m € Z/{0} (2.58) 
= (i2rmy< R ^Xt*+ l) [iJi], m€Zf{0} (2.59) 
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with D~ l [m} = 0 for m = 0 and thus 

2. Similar to the second part in the proof of Prop. 2.2 we expand the inner 
product between the piecewise polynomial signal and the differentiated 

sine sampling kernel defined by ${t) = £ (£2*rm) R+1 . That is, 

the eainple values are 



tt*[t\ = <*«),^{t-rr) >, I=r0 t ... ,r/r-l (2.60) 

r ^ 

- /x(t) J2 {&*tn) A+1 e i *" , *l l -* t V T dt (2.81) 



0 



- £ (iShrm)** 1 e" i ^ rmlT/T / *(t) e< 2 * m >^ dt (2.62) 

N > ^ * 

tX[~m] 

K 

- r ]C *H»J <*27rm)* +i € -o*»VW« , (2 .63) 

Since y 9 [l\ is periodic with period r/T r the DTFS coefficients of are 
given by 

r/T-1 

*iM = 5Z V#P]c"" <asrW/Cr/T) (2.64) 

(2.65) 

A r/T-1 



_ f t/T if ib f rn = 0 
[ 0 otherwise 



~(-t27rm)* 41 X[fc] (2.67) 

Therefore the CTFS coefficienta of the signal are obtained by the DTFS 
coefficients of the samples values V,[m],m =0,... , r/T-1 and are defined 
by 

^NJ~\ 0 form = 0 ^ 2 ' 68) 

Since r/T > 4- 1 the sample values are a sulficient representation of 
the spectral values of the signal. This completes the proof. 
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Note that removing the restriction € C*" 1 leads to the same result as 
in Theorem. 2.1. 

2.3 Aps&Sications 

The applications we consider involve the discrete-time periodic stream of Diracs 
and piecewise polynomial signals. It is well known chat a bandlimited signal can 
be perfectly recovered from its samples by Sampling it at twice the itisudfiaum 
frequency. What if the bandlimited signal has a jump or a discontinuity then 
the signal is no longer bandlimited and the usual method is not valid. These are 
what we call piecewise baadiimlted signals. Another type of noti-bandlimited 
signal which we may come across in nature is a signal which is obtained from 
a system with a certain frequency response. The output of the system is a 
filtered signal. We will look at filtered stream of Diracs and filtered piecewise 
polynotnials. 

2.3.1 Piecewise bandlimited signals 

A discrete- time periodic piecewise handlimited signal is the sum of a bandlim- 
ited signal with a stream of Diracs in the simplest case or with a piecewise 
polynomial signal. An. example is illustrated in Figure 2.6(e) and is obviously 
not bandlimited from Figure 2,6(f). Formally, we have the following 

Definition 2.1 Piecewise bandlimited signals. 

Let Xfi/. 6e a discrtte-Umti periodic L- bandlimited $ignal of period N with cor- 
responding DTFS coefficients X B r, such that X 3 lwi\ = 0 Vm gf L]. Lei 
xpp he a zero mean discrete-time piecewise polynomial signal of period N wiifi 
K pieces and with each piece of maximum degree R. Then a piecewise bandlim- 
ited signal x is defined by 

x^x BL +xp P (2.89) 
with corresponding DTFS coefficients X defined by 

First consider a stream of K weighted Diracs, x^p. From Section 2.1.1, we 
can recover the K weighted Diracs from 2K contiguous frequency values Xpp. 
Since the DTFS coefficients of the bandlimited signal, X^x,, axe equal to zero 
outside of the band [— L, L] t we have that the DTFS coefficients of the signal 
outside of the band [~L, L] axe exactly equal to the DTFS coefficients of the 
piecewise polynomial, that is* X[m] ~ Xpp[?n] 3 V|ro| > L. Therefore it is 
sufficient to take the 2K DTFS coefficients of the outside of the band 
for instance m [L + 1, L 4- 2K\. Suppose we have the DTFS of the signal X{m) , 
with m G [-(L -h 2^), L -f 2K\ then the DTFS of the bandlfrmted signal are 
obtained by subtracting X PS >{m) from X [m] for m € [-L,L]. 
Recall that the piecewise polynomial has 2K(R i- 1) degrees of freedom and 
the bandlimited signal has 2L + 1- It follows that we can sample the signal 
using a discrete-time periodized differentiated sine sampling kernel bandlimited 
to 2K{R tl) + L. 
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Cofroilary 5L1 Consider a. pieeevme bandlimited signal x as defined in Defi- 
nition. &A. 1st tfi[n) be the {R -f T)th differentiated sine mmpling kernel with 
DTFS 

Let M be an integer divisor* &fN, tetd let JVfM > 2{tK{& + 1) 4- L) then the 
samples 

y 9 [t] =< sfn], 0[tt - tM] >, 4 = 0,.-^ N/M - 1 (2.72) 
ere a sufficient representation ofx. 

Pro6fi The proof is exactly the same as in Theorem 2.1 until equation 

Y 0 [k] = ^(l^^)^ 2 X[fe],fc==0,..- ? 2^(ie+l)4-L (2.73) 

Therefore 2/C (ft ■+> 1) values of 

XppM = (1 ,^ )ft ^T V; W . * € [L + 1, 2K[R 4- 1) + L] (2.75) 

are sufficient to recover the piecewise polyaoinial xpp. From these we can 
recover the L spectral components of the baadlunited signal since 

Xstm = ^j^WW -JTjvM), 0 (2.76) 

This gives ua the the bandiimited signal x#l and thus the piece wise bandlimited 
signal as defined in Definition* 2.1 is recovered X = xbl + xpp. ■ 
Figure 2.7 the illustrates the reconstruction of a bandlimited plus a piecewise 
constant signal "using the followmg reconstruction scheme: 

Algorithm 2.1 Reconstruction of piecewise bandlimited signcte- 

Require: N, M, N/M > 2(2JK (R + 1) + + 1; 

Calculate the samples y 8 [i] =< x[n] 5 ^[n - iM] >, 2 =0,,,. , iV/M - 1; 
Ctacti/afe the DTFS X[m]>m € [-{2K{R+1)+LM2K{R+1) + L)} from the 
DTFS of samples y$] — + XppH = X[m],m € [L + 1, (2iT(12 + 1) + £,)],- 
Soteia h * X PS >{rn} « 0, m £ [Z, 4- 1, (21C(J* + 1) + X)] — ► x P p; 
Ccdculate Xbl\™] — - Xpp[m] t to 6 Z»] — ► X££,* 

reconsiratcfrwm ts x =s x S £ + x f p. 



2.3.2 Filtered piecewise polynomials 

Anotber application of sampling piecewise polynomial signals consists in sam- 
pling their filtered output. Figure 2.8 illustrates that a filtered stream of Diracs 
is not bandlimited. These signals are formally defined in the following 
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Definition 2*2 Filtered piecewise polynomials. 

Let xpp be a z&rb mean disctvte-iiifne periodic piecewise polynomial signal of 
period N with K pieces of maximum degree R. Let g be a filter with DTPS Gr. 
Then a filtered piecewise polynomial x is defined bp 

x^g*Xpp (2.77) 

and the corresponding DTFS coefficients X are defined by 

X{m) = G[m] • X P p{ntl m = 0, . . . , N - 1. (2.78) 

Suppose Xpp is a stream of 2<f Diracs. If the filter has IK contiguous nonzero fre- 
quency values G[m] than 2# frequency values of the signal A"[m] will be enough 
to determine 2K frequency values of the stream of Diracs, since Xp P [m] — 
X[m]/G[irri\ y and from Prop. 2.1 these are sufficient to recover the stte&tti of 
Diracs . 

Corollary 2.2 Consider a filtered piecewise polynomial signal x as defined in 
Definition. with Cf[m] £ 0,*n € 1), 4-1)]. Consider an (R+l) 

differentiated sine sampling kernel tp[n] with DTFS 

*t»n] = (flM) W Hwi),«(R+i)|' ( 2 * r9 ) 
Let M be art integer divisor of N sitch thai *TfM > 2K(R 4- 1) -f* X. Then the 
filtered piecewise polynomial signal can be recovered from the NfM samples 

y.ffl-<4rt]^En-~IM]>, *=0,... ,N/M-l. (2.80) 

Proof: Similar to the proof of piecewise bandlimited signals, we have that 
the DTFS coefficients of the samples y 9 [l\ are equal to 

Y m [k] = jjll-W^P+iXlkl fe€ [~K(R + 1),JT(« + 1)] (2,81) 
- £(1 - [G[k] X Pr [k]y (2.82) 

Since G[k] ^ 0 for k £ [-K{R + 1), l)j we have 2jftT(i? -h 1) values of the 

DTFS of the piecewise polynomial 

Xpp[k]= (l-Wft)^G[k] Y ' [kl *S 1),-^ + 1)1 (2-83) 

which are sufficient to recover xpp and which leads to the filtered signal "by 
Definition- 2.2. ■ 
The reconstruction scheme is described in the following algorithm and an 
example of the reconstruction is illustrated in Figure 2.9. 

Algorithm 2-2 Require: N, M> NfM > 2K(R + 1) + 1; 

Calculate the samples y s [l] =< x[n\^[n - IM) > >l ~ 0,. . . , N]M - 1; 
Calculate Y s - DFTjv/m • y, — > X\m\ m e \-K(R + 1), K(R + 1)]; 
Calculate X P p[m] = X[w]/G![m) T m € [-if(H 4- 1),#0R 4- 1)]; 
Solve h * Xpr[m] = 0,m € + l) ? JT(Jl + 1)] — ► x P ^; 

T&e rec«7Wtrttcfion is x = g * xpp . 

We have seen that the crux of the proof relies on the fact that the filter is 
known and is invertibie over the number of degrees of freedom of the problem. 
What if the filter has a unite rate of innovation but is unknown? This is more 
complex and remains to be investigated. 
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(a) 
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(b] 
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(c) 



Figure 2.1: (a)DTFS of stream of Diracs, e [0>iVj; (b) DTPS of low- 

pass approximation Y[k] = 6 [— i£» JST], 0 otherwise; (c) DTFS of lowpass 

approximation subsamplecS by M = 3, Y 8 [fc] — X/MX[fe],jfc € K], 
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Figure 2.2: (a) Periodic discrete-time signal with K = 15 weighted Dlrats with 
period N = 256 ;(b) DTF5 A'[m]; (c) Discrete-time periodized sine sampling kernel, 
(d) DTFS iJect^^u = 15(e) Sample values y a [l] =< a?[n], <p[n-lM] > 
J = 0,. ,31 with Af = 8; (f) DTFS Y„. 



2,3. Applications 



27 




Figure 2 J: Stream of K == 16 bunched Diracs with period N = 64. 



T 




a » ♦ § a r » ■ » ji -a « u 1* a 



Figure 2.4: Comparison between the root finding method and the spectra* extrap- 
olation method on a signal of length N = 64, K varying between 2 and 15 on 
interval 2K, 100 simulations. 
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Figure 2-5: (a) Discrete-time periodic piecewise linear {R ~ 1) signal of period 
N - 1024 with K = 6 pieces; (b] Differentiated sine sampling temel,4>(n] = 
d[«]*«J[ra] *<p[n] with DTPS i?[rn]-Hecfc[_ K(jR+1)iK<JB+i)1 (c) Sample values 
a?[n],^[rc - IM] >J = 0,.-. ,31 with M « 32; (d) Stream of + 1) = 12 
Diracs obtained from X[m] 3 m e [-K{R+ 1), K(R+- !}]> 
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Figure 2,6: (a) BancJlimited signal of length N = 256; (b) DTFS of Bandiimited 
signal, £ ~ 15 (c) Piecewise constant signal with K = 3 pieces; (d) DTFS of 
piecewise constant signal; (e) Bandiimited piecewise constant signal; (f) DTPS of 
bandiimited piecewise constant signal. 
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(c) 



Figure 2.7: (a) BandUmited piecewfse constant signal, a; [n], with K = 3, £ = 

= O t iY = 256; (b) Differentiated sine sampling kernel. %j>[n] =* <f[n] * 
bandiimited to22£(i^l)+l-h2/ = 22 (c) Sample values y a [f] =< ar[»],tf[n-IAf} > 
,2=0,.. , iV/M - l s Af = 4; (d) |jDTFS?) of sample values; Reconstruction error 

is 1G™ 1 *. 
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Figure 2,8: (a) Stream of /C = 4 Diracs with period N = 64; (b) jjPTFS] of 
stream of Diracs (c) Known filter g[n] = o: n ,n = 0,.,. ,iV - I, a = 0.4; (d) 
\DTFS\ of filter; (e) Filtered stream of Diracs; (f) \DTFS\ of filtered stream of 
Diracs. 
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(c) (d) 



Figure 2.»: (a) Filtered stream of Diracs, ar[n], iV = 64; (b) Sine sampling kernel 
ip[n] bandiimited to K — 4 (c) Sample values pjZ] =< x[n] 9 Lp[n ~ iM] >J = 
0, . . . , 15, M = 4; (d) jDTFSI of sainpte values; Reconstruction error is 1CT 13 . 
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Summary 



• We derived sampling theorems for periodic signals in particular streams 
of weighted Diracs' arid piecewise polynomials. These signals have a finite 
rate of innovation p which is equal to the number of degrees of freedom 
per period- 

• The samples are obtained by taking the inner product of the signal with a 
shifted version of the periodized sine kernel or differentiated sine kernels 
The bandwidth of these kernels must be greater or equal to the degrees of 
freedom of the signal- 

• The discrete-time periodic signals are perfectly recovered when the sam- 
pling rate 1/M is greater or equal to the rate of innovation /> =* IKjN in 
the case of streams of weighted Dhacs or p = 2AT(J£4- \)jN in the case of 
a pieeewise polynomial signal with K pieces and maximum degree R. 

• The continuous-time periodic Streams of Diracs and $>iecewise polynomial 
signals ate perfectly recovered when the sampling rate 1/T is greater or 
equal to the rate of innovation p = 2Kft since we assumed that the 
piecewlse polynomial signal belonged to C n l . 

• The sampling and reconstruction scheme is illustrated in Figure 2.10. 
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1 AimihiktiAg 
. J filter method 
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X^ffc], & e {-K(R + 1), if (H + 1)] 



=< 1>[n - IM) > — «- Y.lfc], k e [0, N/M - I] 

Figure 2.10: Sampling and reconstruction scheme for discrete-time piecewise poly- 
nomial signals with K pieces of maximum degree R\ N/M is the nu mber of samples; 
2K(R-h 1) -b 1 is the bandwidth of the sampling kernel. 
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Sampling signals with finite 
rate and finite local rate of 

innovation 

In this chapter 1 we go beyond periodicity in terms of the signals and the sam- 
pling kernels. Section 3.1 investigates sampling finite length signals with a. finite 
rate of innovation using sampling kernels with infinite support. The signals in 
question are streams of weighted Diracs sampled with the sine and the Gaus- 
sian kernel. These types of kernels are appealing to mathematicians. It will be 
shown th&t if the critical number of samples is taken then a sampling theorem 
can be derived. Section 3.2 considers the dual problem: Infinite length piece- 
wise polynomial signals and compact support sampling kernels. A particular 
interest is given to bilevel signals with a finite local rate of innovation and spline 
sampling kernels. Given that the signals have a finite local rate of innovation, 
local reconstruction is possible and schemes are given bi Section 3.2.3. 

3.1 Finite length signals with finite rate of innova- 
tion 

A finite length signal with finite rate of innovation p clearly has a finite number 
of degrees of freedom. The question of interest is: Given a sampling kernel with 
infinite suppcrtj is there a finite set of samples that uniquely specifies the signal? 
In the following sections we will sample signals with finite number of weighted 
Diracs with infinite support sampling kernels such as the sine and Gaussian. 

3.1.1 Sine sampling kernel 

Consider a continuous-time signal with a finite number of weighted Diracs 

Jt-i 

(3-1) 

k^O 

1 Thia chapter includes research conducted jointly with Martin Vetterli and Thierry Blu 
[80, 78]. 
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and an infinite length siti^ sampling kernel, see Figure 3.1. The sample values 




figure 3.1: (a) trample of a finite length continuous-time stream of K = 8 
Qiracs randomly spread on an interval [0, t) with r = 8; (b) Sine sampling kernel, 

sinc(t/r),r = 2. 



axe obtained by filtering the signal with a sine (t/T),t e R, sampling kernel. 
This is equivalent to taking the inner product between the signal and a shifted 
version of the sine 

y n *(* ) , sinc(e/T - n) > (3.2) 

where sfcnc(r-) = sin(iri)/7rt. The question that arises is: How many of these 
samples do we need to recover the signal? The signal has 2K degrees of freedom, 
K from the weights and K from the locations of the Biracs and thus N samples, 
> 2K, will be sufficient to recover the signal. Similar to the previous cases, 
the reconstruction method will require solving two systems of linear equations: 
one for the locations of the Diracs and the second for the weights of the Diracs. 
These systems admit solutions if the following conditions are satisfied: 

[CI ] Rank(V) < tf+1 where t> nJt = A* ((~l)"n* y n ) and V e ^~K)x{K-^i) ; 
[C2 ] Rank(A) = K where a nk = and A € R K * K . 

Theorem 3.1 Given a finite stream of K weighted Dir&cs and a sine sampling 
kernel slnv,(t/T). If conditions [ClJ and [G2f are satisfied then N samples with 
N>2K 

y n = <x(t),shx:(t/T-n)> (3.3) 
are a sufficient representation of the signal 



3.1. Fsttite tettgth signals wittt finite rate of innovation 



3d 



Proof: Taking the inner products between the signal and shifted versions 
of the side sampling kernel yields a set of N samples 

if* = < x(t), aiac(e/r - n) >, *t ^ 0, . . . , tf - 1 (3.4) 

- f J2<**(*-**)«««c(t/r-«)d< (3.5) 

£5 *(tfc/T-n) { *- T) 

- HrgfM (3 . 8) 

The denominator of the previous expression (3.9) ean be rewritten as follows: 
1 _ rW<t/r-n) P fc (n) 

where F(u) is a polynomial of degree K with zeros at all values of tfc/T\ 

ft— v ic 
Pin) = Y[{tt/T-u)=J2p»** (3-11) 

and the P^tt) is a polynomial of degree K — 1 and has zeros at all locations 
except at location t fc 

Therefore if the coefficients of the polynomial P\n) are determined then the 
locations of the Diracs are simply the K roots of P{u). We can now find an 
equivalent expression to (3.9) in terms of the interpolating polynomials: 

1 K-l 

(-l)"F(n)|fc - -^^sin^tfe/TjPfcCn). (3.13) 

Note that the right-hand side of (3.13) is a polynomial of degree K — 1 in the 
variable re, applying K" finite differences makes the left-hand side vanish, 2 that 
is, 

&K((-irP(n)y n ) =0, n = K,„.,N-l (3.14) 

& T£*P*± K (i-l) n n k Vn ) =0 (3.15) 
* f 

Vp =0 (3.16) 



2 Note that the K finite difference operator plays the same role as the annihilating filter m 
the previous chapter. 
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when* the matrix V is an (N - PC) x (iC 4- i) matrix aad admits a solution when 
N-K >K and the rank(V) ia less than K+l>> that is, condition [CI]. Therefore 
(315) caii be used to find, up to a normalization, the K 4- 1 unknowns pk which 
lead to the K locations ik- Otioe the IT locations are determined the weights 
of the Diracs ate found by salving the system in (3*9) fetn^O,... , # t . 
Since tk # t* ^ the system admits a solution from condition [C2]. M 
Note that the result does not depend on 2\ In practice if T \s not eho&en 
appropriately then the matrices V may be in-conditioned. Figute 3.2(a) ilitis- 
fcrates the conditioning of the matrix V is the least for T close to 0,5 and that 
the matrix A is well-conditioned on average. 




W G>> 

Figure 3.2: (a) Average condition number of the matrix that leads to the locations 
of the Diracs, V, versus the sampling interval T, optima! T » 0.5; (b) Average 
condition number of the matrix that leads to the weights of the Diracs, A, versus 
the sampling interval T, optimal 2 1 « 1. Average is taken on 100 signals with 8 
Diracs uniformly spread in the interval [0,8], 



By choosing more adequately the interpolating polynomials, for example by tak- 
ing the Lagrange polynomials, we may reduce the conditioning of the matrix V, 
but thss remains to be investigated. The algorithm is as follows: 

Algorithm. 3*1 Finite length stream of Diracs sampled with a sine sampling 
kernel 

Given y n x{i), sinc(t/T - n) >, n = 0, 1, . . . , JV - 1; 

Calculate = A* ((- 1)* n k y n ) , n = K, . . . 7 N - 1, k = 0,. . . ,K; 

Solve the linear system V • p = 0 — ► {po,Pi> - • - ,Px}; 

Find the K roots of F(u) « £ p fc u k —4 {fc/T, t x /T, . - . , iftr-L/T}; 

Calculate Y n « P(n) y ni n = 0, 1, . . . , N - 1 

Solve the linear system A • c = Y — ► ci, . . . , cj*_i } . 
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This method can be extended to triecewise polynomials, similarly to Theo- 
rem. 2.2. Also, there is an obvious equivalent for discrete-time signals in f 2 {&) 
and discrete- time Sine kernels. 

3.1.2 Gaussian sampling kertid! 

Consider sampling the game Signal as in (5.1) but this time with a Gaussian 
sampling kernel^ <p#[t) ~ e-^ /2er3 , see Figtire 33. Similar to the Sine sampling 



1 



(a) 




Figure 3.3? (a) Example of a finite length contlnuous-tinie stream of K = 8 Diraes 
randomly spread on an interval [0,rJ with r = S; (b) Gaussian sampling kernel, 



kernel, the samples are obtained by filtering the signal with a Gaussian kernel. 
Since there are 2K unknown variables we show next that N samples with N > 
2K are stifftcient to represent the signal. 

Theorem 3.2 Given a finite stream ofK weighted Diraes and a Gaussian sam- 
pling kernel ip a (i) = 

fi -^/2cr» jfM>2K then the N sample values 



Vn ~<x{t) i v e {tjT-1\)> 
ore sufficient to reconstruct the signal. 

Proof: The sample values are given by 



CO 

- / 



fc-0 



fc-0 



(3.17) 

(3.18) 
(3.19) 

(3.20) 



42 



Chapter 3. 



We expand (3.20) and regroup the terms so as to have variable that depend 
solely ott n and solely on fc. We obtain 

^ * ^(cfce^^^J-e^/^.e-^W (3.21) 
which is equivalent to 

where we let Y n ^ e "*f 2 * 2 y n$ a* = c A e-*!/ 2 ^ 2 m d tik — e t ±f tr * T . Note that 
we reduced the expression F rt to a linear combination of real exponentials^ This 
hints that the amiihiiatirig lilter method described in the Section 2.1.1 seeing 

appropriate to find the K values a*. Let H(z) ho 4- ht z' 1 H h he 

an annihilating filter, that ia, h is such that 

h * Y ^ 0 (3.25) 

& =0. ft^HT,... ? iV-L (3.24) 

Note that this is a Toeplite system with real exponential components K» — 
e n 2 /2<r ^ atici therefore a solution exists when the number of equations is greater 
than the 1 number of unknowns, that is, N — K > K and the rank of the system 
is Less than K -f 1 which is the case by hypothesis. Furthermore & mu^t be 
carefully chosen otherwise the system is iU-cottdit.ioned* tf we factor M{z) = 

f[ ( 1 — ^~ x Wfc) then we obtain the locations of the Diracs th from the roots of 

fc=G 

the polynomial H{z), that is, 

t k ^ a^Tluu^ (3.25) 



Once the values of the Diracs t& are obtained then we solve for at the Vander- 
monde system in (3.22) for which a solution exists since u& u;,V& J. The 
weights of the Diracs are simply given by 



(3.26) 



Here unlike in the sine case, we have an almost local reconstruction because 
of the exponential decay of the Gaussian sampling kernel which brings us to the 
next topic. 

3.2 Infinite length signals with finite local rate of 
innovation 

In this section we consider the dual problem of Sec. 3.1, that is, infinite length 
signal* x(t) t t € ffit with a finite local rate of innovation and sampling kernels 
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with compact mppcrt. In particular, the 0— splines of different degree d are 
considered [73] 



where v? 0 (0 is the box spline defined by 

_ f t ifO< 
= \ 0 else 



t< 1 



(3.27) 



(3.28) 



We develop local reconstruction algorithms which depend on moving intervals 
equal to the ske of the support of the sampling kernel, 3 The advantage of 
local recoiisLt- action aigorithniB is that their complejdty does not depend on 
the length of the signal. We begin by considering brieve! signals , followed by 
piecewise polynotnial signals. 

3.2.1 Bitevsi signal* 

Consider an infinite length cohtinuous-time Signal st{t) t t € Ife 4 * *hich takes on 
two values, 0 and 1. with initial condition x(i)\ t ^ = 1 with a finite local rate 
of innovation, p. these are called hllevel signals and are completely represented 
by their transition values t k . For example* binary signals such as amplitude or 
position modulated pidses or PAM, PPM signals [31], see Fig. 3,4. 



\ 1 r- 



,ji 

S 



1 8 



Figure 3.4; BiSevel signal 



Suppose a bilevel signal is sampled with a box spline <p 0 (t/T). Then the sample 
values are giveit by the inner products between the bilevel signal and the box 
function, 



irn=<*(0,Y*(*/T-n)>= f x(t)<po(t/T-n)dt. 



(3.29) 



It can bee seen in Fig. 3.5 that the sample value y n corresponds to the area 
occupied by the signal in the interval (n -f l)T], Thus if there is at most 
one transition per box then we can recover the transition from the sample. This 
leads us to 



s Tbe sise of the support aC tp d [tfT) is equal to {d +■ 1}T. 
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-i 1 — r~ 



8 



(b) 



Pigpfti 3.5: (a) Box solium sampling kernel, <po(t), T 1. (b) Blieve! signal sampled 
with the bo* sampling kernel. 



Proposition 8*1 A bilevei signal x(t)^t > 0, ndth initial condition x(t)\ is=fS - 
1, is uniquely determined from the samples t/ n ^< x(*) ; tpo(t/T - n) > tti/ie*« 
<£»o{£) is tAe tos spfttte defined in (3. S3) if and oniy if there is at ih&st one 
transition in each i&tefrml [nT„ (n 4- 1)2*]. 

Proo/; For simplicity let T = L Consider an interval [rt, n + 1] atid suppose 
— 1. First we show sufficiency followed by necessity . 

: if there are 0 transitions lo the interval [n,n + l] then the area under 
the bileval signal, or the sample value, is y n = i since we supposed that 
x{t)\ t „ n — 1. If there is one transition in [r^rt 4- 1] then the sample value 



is equal to 



y n = < ^(t), ^o(* — >= j %{t) {t — n) dt 
= / x{t) dt - J ldt=t k -n 



(3.30) 



(3.31) 



This implies that tk = y n — **• Similarly if x(n) = 0 then we have £*. = 
n 4- 1 — y^. Therefore we can uniquely determine the signal in the interval 
[n,7i+ 1], 

Necessity is shown by counterexample. 

Suppose x{n) = 1 and there are two transitions in the interval 

[n, Ti+l] then the sample value is equal to 



= i k - n + n 4- 1 - t* + 1 = £ fc - * fc+ i 4- 1. 



(3.32) 



(3.33) 

That is, there is one equation with two unknowns and therefore insuf&cient 
samples to determine both transitions. Thus there must be at most one 
transition in an interval [n, n + 1] to uniquely define the signal. 
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Now consider srnftirig the bilevel signal by an unknown shift €, see Fig. 3.6, 
then ihens will be two transitions in an interval of length T and one box function 
will not be sufficient to recover the transitions- Suppose we double the sampling 




t 



Figure 3J: Shifted bilevel signal with two transitions in the ihterva! [5,6]. 



rate, then the support of the box sampling kernel is doubled and we have two 
sample values y nt p*t+\ covering the interval [*iT, (« + 1)7'] but these values 
are identical (see their areas). Therefore increasing the sampling rate is still 
insufficient. 

This brings us to consider a sampling kernel not only with a larger support but 
with added kdbrmation. £br example, the hat spline function ipi[t/T) defined 
by 

««)-{ *-w »W<> (3 . S4 , 

leads to sample values defined by y n —< x{t) , tpi {t/T - n) > or 

»T (n+l)T 

g« - y* ir(t)(l4-t/T«7i)d:t+ j x(t)(l-(t/!f -n))<fc.(3.35) 

Prom Fig. 3.7 we can see that there are two sample values covering the interval 
[nT f (n + 1)TJ. We will show next that in this case we can uniquely determine 
the signal. 

Proposition $.2 An infinite length bilevel signal x{t), with inittai condition 
x(0) = 1 ts iwj^iid^ determined ^/rom ifre samples defined by 

y n = < *>i - n) > (3.36) 

wftere 43 toe feat sampfctt^ fcemd «/ and only if there are at most two 

transitions t& ^ tj in each interval [tiT, [n + 2)T]. 

Proof: Again, for simplicity let T = I and suppose the signal is known for 
t <w and z{t)\^ = 1. 

First we show sufficiency by showing the existence and uniqueness of a solution. 
Then we show necessity by a counterexample. 
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figure 3,7: (a) Hat spline sampling feirneLpP/T), T = 1. (b) BHevel signal with 
t\tfd transitions in an irtterval [n<n 4- 1] sampled with a hat sampling kernel. 



SintU&t to the box sampling kernel the sample values will depend on the 
configuration of the traditions in the interval [n,n + 2]. If there are at 
most 2 transitions in the interval fn, ft 4-2] then the possible configurations 
are 

(0,0),(0 1 1),(0 I 2),(1 # 0) > (1 > 1) 1 (2 # 0) 

where the first and second component Indicate the number of transitions in 
the intervals [n,n+ 1], [n 4-1, ft 4-2] respectively, see Fig. 3.8. Furthermore 
since the hat sampling kernel is of degree one we obtain for each configu- 
ration a quadratic system of equations with variables t 0i *i. 

n n+1 
Vn « y (1 + * - 4- y* (1 - (t - n» (3,37) 

n— 1 r» 

S/ n ^ L = J aK*)(l + i-(n + l))# 4 y *(0(l-(t~(n + l)))cft. 

(3.38) 

First we show that the quadratic system of equations admits a solution 
and then that it is unique. 

(a) Existence. 

Take n — 0 and so the moving interval is [CL 2] . 

The configuration (0, 0} will lead to sample values y 0 = 1, y\ — 1. 

The configuration (0, 1) will lead to sample values 

y 0 =l/2 4-J(*-l)ift =|*o 2 + l-*c (3.39) 
i * 

yi = l/2 + /(2-*)<ft -it 0 2 -l4-2* 0 (3.40) 
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Figure 3.0: BUevel signal containing at most 2 transitions In the interval [0, 2\: All 
possible configurations. 



=^ *o « ifo 4- yx = 1 + v 7 " 1 + 2 Bo = 2 - 72 - 'iiJi. 
The configuration (Ov 2) will lead to sample values 

y Q = + + (3.41) 

in - ~|^o 2 + l+2fo+it 1 2 -2t 1 (3.42) 

*0 ~ 2(-Ao-i-*>) *** 2(-2lio+vO ' 

Tlie configuration (1, 0) will lead to sample mlues 

Vo = ~^o 2 + * 0 (3.43) 

Vi - ~*> 2 (3.44) 

=> *o — vo + yn = i >/i — 2^o = V^yT- 
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The configuration (i^l) will lead to sample values 

Va = -^ + *o-£*i 2 +ei (3.46) 
ft » |t 0 3 4-2 4-i<i 2 -2ti (3.46) 

_^ ^ jjo-hyi 4- -^mk 4-4$ i -gj ^ ^ -fyp+ 44- V-aj -3»»tti 4 4y , 

Hie configuration (2,0) will lead to sample values 

Po -j*o* + 1 + *d+ j*t a -ti (&47) 
yt - jtf-M-Jti 9 (4-48) 

^. j. _ 3-^j/i-h^?4-iitfoj/i4-^8-4tr p * _ 6- 4frq -6pi ftf?4j yq yi 4-fl§ 

(b) Uniqueness. 

if p n = 1 And ^ti+i = t knen this knpiies configuration (0,0) . 
If y n s= 1 and 1/2 < jfo+i < 1 then the possible configurations are 
(0, 1),(0,2). By hypothesis, there are at most two transitions in the 
interval [n 4- i*n 4- 3] therefore if y„4i < 1/2 then the configuration 
in the interval fn,n 4- 2] is (0, 1) otherwise if #n+2 > 1/2 then the 
configuration te (0,2). 

1/2 < # n < 1 and 1/2 < |/ rt 4*i < 1 then this implies configuration 
(2,0). 

U 1/2 < y n < I and 0 < 2^4-1 < 1/2 then this implies configuration 
(1,0). 



>z Necessity is shown by counterexample. 

Consider a bilevei signal with three transitions in the interval [0,2] but 
with all three in the interval [0, i], see Fig. 3.9. Then the sample values 
in this case are are equal to 

Ito = X/2 + f{l-t)d£ + j(l-t)dt (3.49) 
o ti 

m 1/2 + to - *i + h ~ *5/2 + tf/2 - t\f% (3.50) 

Vi 

o t x 

= *o/2-*!/2 + «£/2. (3.52) 

There is no unique solution for this quadratic system of equations. There- 
fore there must be at most 2 transitions in an interval [0,2], 

■ 

Once again if there is an unknown shift in the bilevei signal then there may 
be three transitions in an interval [n2\ (n -f 1)21 and so we increase the number 
of samples by sampling with <p x (t/(T/2)). The pseudo-code for sampling bilevei 
signals using the box and hat functions are given in full detail in Section 3-2.3- 
When going to higher order splines, necessity carries over. Sufficiency is more 
tedious since we must solve a system of higher order polynomial equations. 



£0 *2 

= ftdt + f tdt (3.51) 
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Figure 3-§2 Bilevel signal containing three transitions in dn interval [4,5], sampled 
with the hat sampling ternet tyi{t). 



3.2.2 Ptecewtes polynomials 

Similar to bilevel signals we cohsidet sampling piecewise polynomials with the 
box sampling kernel. Consider an infinite length piecewise polynomial signal 
${t ) where each piece is a polynomial of degree R and defined on an interval 



*e [o,i 0 ] 



(3.53) 



Each polynomial piece Xfe(t) contains jR + 1 unknown coefficients c^m. The 
transition value t k is easily obtained once the pieces x*-i[t) and x k (t) are 
determined, thus there are 2(R + 1) + 1 degrees of freedom. If there is one 
transition in an interval of length T the maximal local rate of innovation is 
p m (T) {2(R + I) + 1)/T. Therefore in order to recover the polynomial pieces 
and the transition we need to have at least 2{R +l) + l samples per inter- 
val T. This ie achieved by sampling with the following box sampling kernel 
^W ^i)^ ) 1 For exam P le if i$ a piecewise linear signal with 2 pieces as 
illustrated in Fig. 3-10 then to recover the signal it is sufficient to take 5 samples: 
two before the transition, two after the transition arid one sample covering the 
transition. 

We can generalize by noting that the Rlh derivative of a piecewise polynomial of 
degree His a piecewise constant signal. The pseudc-code for sampiing piecewise 
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Figure 3.10: Ptecewise linear signal sampled with a box sampling kernel. 



coiLSt&nt signals with the box sampling kernel is fatind in Section 3.2.3 . 

3.2,3 Local reconstruction algorithms 

The following algorithms have been implemented in Maple™. In all of the 
algorithm* k is the hide* of a transition value and n is the index of the current 
interval fn 4 n + Ij. We suppose that x(t) = I,V£ < 0. 

BileveE signals 

Suppose N sairjjle values ff* =< a?(t), y*>(t — n) > are available. 

Algorithm 3 jtf BUevet signal with box sampling kerrieL 
Require: k 0, n <- 0 

= l Vfe[n f n + 1]. 
end a/ 

*/ If* ~ 0 t/*e« 

ar(0=^0 Vt€ {n,n+ Ij. 
end i/ 

i/0 < y tl < 1 tfaen 
if = 1 tfcen 

«fc <- 3/n + n 
else 

t k n + l-$n 
end if 
k <~ k + 1 
end */ 
n «— n + 1 
end ttf&iJe 
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Next we give the pseudo-Code for bilevel signals sampled with a hat sampling 
kernel Suppose N sample values y n ==< ar{£) t <pi(* - rt) > are available. The 
variable tncode is a set whose last component indicates* the tmtaber of transitions 
in the interval [n - 

Algorithm S Jf Btieiid signal with hoi sampling kernel 

Requires tncade *- 0, & *- 0, n < 1 

t_*/*£te n < N -I do 

incorfe 4— 0 

V<€ [tt- l,n + 1] 

end if 

tfttti^® then 
tncode 4- 0 

end if 

ifO < Vn<l £&ert 

ifO iransitiom in the internal [n - tften 
*/ ?/n = 0-5 then 

tncode <- 0 

so* = ^of^e /or cortfiguratim (l» 0) £ [H, n -f 2j 
tncade 4- 1,0 

else 

tfc n -r 1 — \/2 y^T 
end i/ 
nf-n + 2 
fe*-fc + l 
aolfound 4- IVue 
else 

sol = sofoe /or cdrc/fc-ro&ott (1, 1) 6 [n s » -f 2] 

tncode 4-1,1 
i/ar{n + ) = 1 tftg n 

+ \/~3 + 2^+i + 2^2-2^ + 2# w 

tfc^nrl- v^vfe 

end if 

solfound 4- 7V_e 
else 

aolfound 4— False 
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end </ 
£rad 
end if 

if not sotfound then 
ml ±* 3oh>& for configuration (2,0) £ (tt,n + 2l 

fcncode 4-2,0 

* . i 3-2 jf n +! -4 fr. +23*41 y n 4-1^4-^^-4 tt42fr» rt-fSgn+i t* 

i 1 3 -6 y^+i 4-4 -a a* tHryg +2 » w +: fru -2 jk+j ^4-^4-x 

**+i 3 -2+&i+i+tM 

. , l zl ^4t-2 it+$l 4 2 Dh+i y„ -2 tr» -i-i ft4-ftjUi 

f * ^ 5 !/»+#*+! 

end if 

k+-k + 2 
end if 

erid if 

eUe ifl transition in ike internal [n — l, n] ihen 

sal = s<jfve /of configuration (1,0) € [rt — 1, n+ i] jitten £fc_i e [n— 1, n| 
if$oi£$ then 
hncode <r- 0 
+- ri -f 1 

sol sojt'e /of cem/toumtion (1,1) € [ft - l,n 4- l] given tk-i € 
[ft - 

if sol *hen 
tncode 4- 1 

*((rt-l) + ) = 1 tfien 

else 

end 

n4-n + l 
fr«-A:-hl 
end*/ 
find*/ 
eke 

{ 2 trajisttium in the interval [n — l,n]} 
tncode 4- 0 
end */ 
end! */ 
end whtie 



Piecewise constant signal 

We consider sampling a piecewise constant signal with the box sampling kernel, 
doubling the sampling rate is auiBcient to recover the signal, thus we suppose 
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2N B&mple values tf tt are available. 

Algorithm %Jt Piecewise constant signal with box sampling kernel. 
Requite: , *- 0,ri D « 0..-2JV - 1 
white n < 2,V - 2 do 

-ir*l = o ***** 
ft « n -t- 1 
else 

J*— « + t 

n n 4- 2 
end 
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3.3 Summary 

• A finite length stream of K Diracs can be recovered from N samples £n 
obtained as the hitler product between the signal and shifted, versions of 
the sine and Gaussian sampling kernel, when N > 2if . 

• For both types of sampling kernels two systems of equations must be 
salved: the first system is to find the locations of the Diracs and the 
second is to find the weights of the Diracs. 

• When sampling a randomly spaced stream of Diracs with the nine kernel 
the system leading to the transitions may be ill-conditioned if the sampling 
interval T dot chosen appropriately. It is illustrated that at critical 
sampling that i$, when we have PC = %K sample values, the optimal 
sampling interval obtained for these type of signals kT==0.5. 

• When sampling a randomly spread stream of Diracs with the Gaussian 
kernel the conditioning of both systems depends also on the value of the 
variance cr 3 in the Gaussian kernel. 

« In the same fashion as in Chapter 2, the sampling schemes using the sine 
and the Gaussian kernels can be generalized to both contmuous-time and 
discrete-time piecewise polynomial signals. 

• Infinite length signals were sampled using a compact support sampling 
kernel. 

• Bilevei signals can be recovered using a Box sampling kernel Va(t/T) if 
and olily if there is at most one transition in each interval [«, (n 4 1)T]. 

» Bilevei signals can be recovered using a Hat sampling kernel ip\{tjt) if" 
and only if there is at most two transitions in each interval [n. (n 4- 2)T]. 

• In general, to recover the intmite length biecettise polynomials with K 
pioces of of maximum degree R using a box sampling kernel, the sampling 
rate must be greater than the maximum local rate of innovation 

A*(r) = (2(#+i) + i)/r. 

ft Sampling and reconstruction algorithms were given for each problem in 
their respective sections. 
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Sampling of Piecewise Polynomial Signals 

Martin Veiterii 
Audiovisual dommtmicatioM Laboratory 
Swiss Federal Institute of Itedmology 
Lausa&ae, Swits^rland 

December 1999 
A,batz*£t 

tex this note* we consider the problem d£ aampliiig piecewise poiy- 
noiaiaL signals using smooth sampling kernels, A few results and a 
mifliber of cettajeciuree axe shown regarding the possibility to exactly 
reconstruct piecewise polynomial signals from appropriate sampling. 

1 Introduction 

Sampling of baadlimited signals is a standard signal processing procedure. 
However, many signals in practice are not baadlimited, and bandlimiting 
thena first might distort them unnecessarily. 

Bandlhnited signals have the property that, unlike general signals, a 
countable set of pat&matere is sufficient to specify them (e.g. the Fourier 
coefficients at thfc periodteed spectrum). There axe many other signals be- 
sides baadlimited ones that are specified by a countable aet of parameters, 
BJid we consider hereafter one such class, namely piecewise polynomial sig- 
nals. 

We pose the problem of finding the parameters of certain piecewise poly- 
nomial signals through regular sampling with a sampling kernel. Topically, 
the sampling kernel is a smooth function with a support at least as large ae 
the sampling interval, (e-g* box or hat function). 

Thus, we have access to the inner products between the signal and shifted 
versions of the sampling kernel. The question then is now to recover the exact 
parameters of the signal. 
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Figure 1: Bilevel 0/1 Signal. 



Figure 2: Transition at to and box function. 



The applications of these methods can be found m signal processing fox 
communications and g&mpUng of images with special characteristics (e.g. bi- 
level images). 

The algorithms involved involve eitfaet the solution of systems of non- 
linear equations, or iterative solution methods. 

These notes are an outline of the problems, some elementary results, a 
set of conjectures* and possible extensions. 

2 An elementary example: bi-ievel signals and 
spline kernels 

Consider bWevel signals, vr.l.o-g {G> 1} signals, with' transitions at arbitrary 
times i see Fig. 1. 

Clearly* they are not bandlimitsd, and thus sampling with a 5(t) kernel 
will not work, ConsideJ: sampling at integers using a box spline centered at 
0: 



Further, assume that x(0) = 1, and that x{t) has at most 1 transition over 
any interval [t , £-f- 1], t € or that transitions axe spaced apart by a distance 
d > L Consider the set of samples 



Result 1: x[n] specific uniquely x(t) (with some initial conditions on x(t)) 

Proof: (Sketch) The value of &[n] gives immediately the transition lo cap- 
tion. Consider t = 0. Then t 0 = x[n] — 1/2 or == — to- 

However * there is uncertainty about $o 3 £q. Thus, we need initial condi- 
tions, like x(i) is equal to 1 from —1/2 to 0 for example. Once the first t 0 is 
specified, the other follow similarly 

If there are more transitions, the sampling is not sufficient, e,g. see Fig, 
3. ^ ' 




z{n] s=< <p 6 (t - n) y x\t) > 



then 
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Figure 3: Uncertainty with transitions that are too close. 
Figure 4: Shift variance. 

Note: The restriction of mo&t 1 tit^titoa iri any intern! % i +1] is more 
than what is really Kieoded. It ia snMciekt to have at mmt 1 traasitioit in 
thfc int&rv^{£— 1/^^ But thfift/t&fe origin 

of tima becomes oritus&l, $ad the sampling condition is not shift mtoi^tit 
anymore e.fr see Fig* 4 

tJsing a ss&apling kemdi that is of wider spread allows to r^lax tile restric- 
tion on the nttinbsr of trat^tidtiB: an the one hand* thfc "average ntu&fcer of 
traja&ition" or 'totoaitian rate cannot exceed 1 (per interval), but dn the 
other hand> it must hold only over the spread of the sampling kernel* not 
over &&ch length 1 interval. Consider the hat function: 



*»-«"" I" 



and 2 transitions over the [0, 1] interval (again assuming tit) 1, t e [-1, 0] 
and t € [l, 2j as boundary conditions). 

Then: 



c{0] ==< ¥>i(t),a:(t) > - 1/2 + / (1 -t)dt + / (1 - 

= 1/2 + to - <o/2 + 1/2 ~ ii + 1?/2 



=< ^(t - 1), *(<) > = 1/2 + f id* + f fcfe 

= l/2 + ^/2 + l/2-tf/2 
= 14-^/2-^/2 

Note: x[0] +- x[l] = 2 + f 0 + *i- 

So, by solving this 2 by 2 non linear system, we can find i 0 »ii» from 
s[6], z[l]. 



Figure 5: Hat function and two transitions. 
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Conjecture 1: For the linear spline case* if ure hews intervals and at most 
2 transitions on intervals of length % thfcn the transitions can be computed 
uniquely (with adequate boundary conditions on the left and right side). 

Clearly more than K trauMtioii^ caniiot he recovered* since we have K 
measurements only. More than 2 transitions in an Interval + should 
lead l td toihigtdty (need to be shown* obviotfcs in box spline c&se). 

TEhe above result can certainly foe extended to splines of afbteary order: 

Gonjedtore 2: For splines of order N> there can be at most N + 1 tran- 
sitions in intervals [n* n + JV + 1]. so 1 the transition times can be exactly 
recovered. 

Remark: as N -4 ao (e-g. Gaussian kernel) we get something like the 
Kadec -i/4 theorem, since the average rate has to be 1. 

Note: recovering {£*} becomes increasingly difficult, since it involves NL 
equ* with if. 

Question 1; What happens if the kernel used in the sampling is the sine 
kernel? FbUon^ing Oonjecttire 2* we should be able to recover the tj's just as 
well, Bttt is there an interesting method* or on the contrary, is the problem 
absurdly difficult because of long range interaction. 

Question 2: la there an iterative algorithm, in particular one that has good 
convergence properties? For example* in the example with linear splines, can 
we ^efficiently* search to, t u e.g. 



While we know there is a solution that satisfies the measured values, the 
problems look more like root finding, and might get stack in local minim as. 
This needs investigation. 

3 Piece wise constant signals 

How can we apply the previous ideas to signals that are piecewise constant? 
Aatonishingly^ it suffices to double the sampling rate! 

Result 2; If there is at most 1 transition in the intervals [n, n + then 
sampling with the box spline is sufficient to reconstruct x{t). 

Proof: (sketch) There are technicalities to set up [x{t) ; boundaries?) 
(e.g. when the signal is zero versus non-zero). Also, requiring the interval 



(given by linear splines) 
(given by linear splines) 
(to be found) 
(to be found) 
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Figtdrfe 6: Piecewise constant signals 

Figure 7: Multilevel sigji&l* 

to be a running interval (t, i 4~ 2) might make it simpler. la any case, one of 
the box splines ''measures* the value* while the other derives the transition 
point: For example see Fig* 6 

Vo-<<Pd(t),*(*)> 

V 4 **<^(t-2),*(i)> 

Then: tc can be derived from Vt and H> ^« 

Oo&jecfrurfe 3: Hie above result can be extended to arbitrary spline ker- 
nels, with necessary adjustments 

In that cme, the same * relaxation* happens as in the bi-level signal, that 
is, only a. bigger interval fat 4- 2N) eaters into the picture. 

In this multilevel cas% we can consider 2 specific cases. 

i) Multilevel with specific levels* e.g. the consts&t pieces! have integer 
values (See Fig. 7.) This should lead to potential Mmplifications, but 
prob, not to a lowering of the sampling rate necessary to reconstruct. 

ii) Piecewise constant signal with noise, potentially with noise levels de- 
pendent on value. This is more realistic, and more difficult. A detailed 
analysis on error propagation ie necessary. (See Fig. 8). 

4 Piecewise Polynomial Signals 

Take the simplest case: 

i) signals are piecewise linear and continuous 

ii) the sampling kernel is the bcoc spline 

Then, with sufficient oversampling, one should be able to recover the signal. 

Figure 8: Multilevel signal with level dependent noise. 
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Figure 9: Piecewise linear signal, and tw6 box splines measuring a linear 
piece. 

Figure 10; Piecewise linear signal, where discontinuities can be found* 

GorLjecttire 4: If there are lesg thim 1 ttfaa&ticm per interval of length 4> 
then the piecewise linear signal can be recovered from bm. spline sampling. 
Sketch of idea : see Fig.. 4L 

There &re at least 2 splices inside each linear piece* These allow to mea- 
sure the linear piece exactly. However, we are not sure where the breaks are. 
With the restriction above, there will always be 3 box splines on a linear 
piece* thus leading to &{t) , a[i + 1], + 2] that are discrete-time linear- This 
allows to test where the breaks are [${k] which is not aligned)* see Fig. 10. 

Here: & * + M '+ 2] are aligned, but (i-l^i+i] and [*+l t l + 2,< + 3l 
are not. : 

C onjecture 5; Using higher order splines relaxes the transition to be within 
a longer interval. 

Oonjectttre 8: Similar results can be developed for piecewise polynomials 
of higher degree, with singularities (discontinuity). Appropriate "oversam- 
pling* is needed* (Enough sampling kernels "inside*' to specify the piece). 

For example, the method developed in Cfefijectttre 4 allows piecewise lin- 
ear signals with discontinuity, since each linear piece is specified indepen- 
dently. Then 3 the sample of the overlap — 1] in the example) can be used 
to specify the discontinuity, or in a diagram. See Fig. 11 (a). 

The value of the "overlap* 5 sample specifies if the transition is as in Fig . 
11(b). 

With these results, we have a ''sampling theory* for piecewise polynomial 
signals. 

5 Extension to two dimensions 

Clearly, a very interesting case appears in two dimensions, where biJevel 
images are quite common (e.g. the woodcut picture). 

The two-dimensional case is however much more involved- Singularities 
are now of dimension 1 (rather than 0), and so cannot measured through 
sampling, unless they ate themselves restricted. 

Figure 11: (a) Finding the discontinuity location (b) Finding the exact value. 
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Figure 12: (&) Arbitrary boundary (b) Smooth boundary. 
Figure 13: Banrilfmited boundary function. 
Oonaider fcha following two eases iti Fig* 12: 

- (a) Unit square* with 0/1 tta&Sitioti given by arbitrary function and 

- (b) tJnit gqii&re, with 6/1 transition given by smooth (e.g, tt b&udlim- 
ited") function. 

OIearly r th6 method developed in the previous sections can be Applied on 
a set of lines {e.g. a square grid) t and it -will work fat case (b), but not for 
case (a) a priori. 

Qofljecfcafre ?: By reacting the transition or singularity to b£ bandMik- 
itfcd, one can give m exact sampling using separable spline kernels. 

Sketch of obvious case: assume a boundary Amotion that is a bandliinited 
function {not & su^ve, but a fun&tion), that is> on m infinite hx&uj*fi } me Fig- 
13. =' 

Apply the spline method on vertical lines, at sampling intervals T < 
This gives exact samples of .the boundary function, which can now 
by perfectly ir&erpolated using Shannon's theorem. 

Similar results can be applied if the boundary function is piecewise poly- 
nomial. 

Conjecture 8: If the boundary function is piecewise polynomial (with a 
minimum separation between transitions) there is a sufficiently fine separa- 
ble sampling using spline kernels that allows a perfect representation of the 
boundray, and thus of the bl-level image. Example: piecewise linear bound- 
ary, see Pig. 14 

The last results can most probably be generalized to: 

Conjecture 9: If an image is piecewise polynomial with boundaries that 
are either bandlirnited or piecewiee polynomial, then separable sampling us- 
ing spline kernels can give a complete characterization if sampling is fine 
enough. 

Note: a bandlimited assumption on contours is not unreasonable in our 
example. 

Addendum to notes 

So far we have considered the two dimensional case in a separable fashion, 
which is too limited. Instead^ it is of interest to consider the sampling of two 

Figure 14: Piecewise linear boundary. 
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Figure 15: Two-dhnensional bWevel signal. 

Figure 16: Intersection of boundaries 

dimensional piecewise polynomial sigfialftj where the sampling leenlei is a non- 
separable point spi-ead function as is* the case in a st real" sampling system (e.g. 
a scanner)* 

Cot^e<Aure 10: 

A bi4evel twcHdimen&onai signal with piecei&Hse linear boundaries of 1 'suf- 
ficient sep&e&tioa* 1 can be perfe<jtiy recovered from a sufficiently <tense^ sam- 
pling with, a separ able or non-sep&r&bie ketneL 

The idea of the proof is elementary* however, the various cases and details 
axe tricky So -we oaly discuss the idea here. Consider the following bi4evel 
two dimensional signal: 

For simplicity, ^ consider the square booc function as the senapling kernel 
(it is separate* but we do not use its separability in its solution). 

Call Sij the valua of the "sample" corresponding to cell Assume the 
boundary is given by y = ax 4- b. Then a?i (1 — fy/a^x* = (2 - 6} /a and 
= Then it follows that 



S 0I = (3-6)/2a (2) 
This leads to a quadratic equation for b 

b 2 - 26(1 + fibo/fitli)ll - 2Sq»/Soi (3) 

where the positive real root is the solution. Then a follows from 1 or 2. 

Clearly things are more complex, since one needs to find two cells on 
the boundary, but it is also clear that a similar method works. In the case 
of more general point spread function, numerical integration is needed, and 
probably a look up table using pairs of values can be used* 

What happens if boundaries intersect? If enough cells are found that 
define the boundaries, then the intersection is defined exactly as shown in 
Fig. 16. 



1 This notion, whldbi is intuitive, needs to be formalised. 
36 Again, notion needs to be made exact. 
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in una case, there are enough cells outside the "intersection cell** tha,t 
defined the boundaries and the intersection. To define exactly the density of 
celts versus the maximum number of individual boundary elements is one of 
the open challenges, even if we intuitively see when it will work and when it 
won't. 

6 Relations to other constructions 

i) K&dec'e —1/4 result: there is probably a relation to this result. In 
particular, as the support of the sampling kernd goes to infinity, only 
an average rate constraint remains, just as in K adeems result on irregular 
sampling 

il) usual ginc sampling: we have only hinted of the relation but there is a 
connection* &ud again, since the support of sine is infinity we probably 
get an "average rate** t&iult. Bo*ravet, sine based sampling i§ much 
more complex. 

iii) Logan's theorem on aero crossings Aga£n> it is a result on b&ndlimitad 
signals, so there is no direct connection. However, it might be interest- 
ing to see if zero crossings, which define polynomials, can be used to 
define piecewise polynomials as well. This is unclear, since breakpoints 
are not j&ero crossings in general. 

iv) Derivative sampling, integral sampling: it is of interest to see if our 
results carry over to derivative and integral sampling. A priori^ consid- 
ering the bWevel signal case, the signal and its (generalized) derivative 
carry the same information, so sampling one or the other, or alterna- 
tively one and the other, gives the same information. 

v) Radon transform: in the context of fai-level images with smooth bound- 
aries, characteristics of the Radon trausform need to be looked at (is 
the Radon transform smooth, can it be sampled etc.) . 

Conclusion 

The above sketched results should serve as a motivation to fully under- 
stand this problem. 

More generally, we conjecture that any object with a countable set of 
parameters that are sufficiently spread out can be recovered by sufficiently 
fine uniform sampling. 
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2.A Annihilating filter method 35 

Appendix 2, A Annihilating filter method 

The problem in spectral line analysis consists in estimating the frequencies of a 
sinusoidal signal from a set af values. The methods used for estimating such fre- 
quencies are known as high-resolution methods, for example MUSIC, ESPRIT 
and can be found in [67]. We define the following as the annihilating filter 
method: 

Consider a signal s[n]»rt e % defined aa a finite lineat combination of K expo- 
nentials ttjjj, 

K-i 

5 [tt]= 5% fe ttf (2.84) 

where c ft are real And u k ate real at complex valued. In the context of spectral 
line analysis u* = e^ fc where is the fcth frequency component of the signal 
sin}. 

Defttiition 2,3 A fitter i - z~ l Uk i& edited an annihilating filter for u% if [61 j 

(i - z~ x u k )u% = 9. (2.85) 
where z~~ l in a shift or delay operator* 

Suppose that Uk ^ 0, then z — satisfies (2.85) and is a zero of the filter 
l-z~ l u k , If there are K exponentials then there are K alters i - z~ % u k each 
annihilating their respective u£. This implies that the product of these filters 

K-i 

for sure annihilates each exponentials, wf, thus H{z) is an annihilating filter of 
e[n], 

s[n] . B(z) s 0. (2.87) 

Therefore to find the values u k we need to find the filter coefficients in 

fc 

H(z) = X>**~* (2.88) 

such that (2.87) is satisfied. Substituting H{z) defined hi (2.88) in (2.87) we 
obtain 

f>*sWs~*-0, (2.89) 
which equivalent to the following recurrence equation 



K 

I 

«s=0 



^feftsln-jfe] = 0 (2-90) 
h*s = 0. (2.91) 



# y 



